<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=11"/>
<meta name="generator" content="Doxygen 1.9.4"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>Flow: net_flow/detail/low_lvl_io.cpp Source File</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr id="projectrow">
  <td id="projectalign">
   <div id="projectname">Flow<span id="projectnumber">&#160;2.0.0</span>
   </div>
   <div id="projectbrief">Flow project: Full implementation reference.</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.9.4 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
var searchBox = new SearchBox("searchBox", "search",'Search','.html');
/* @license-end */
</script>
<script type="text/javascript" src="menudata.js"></script>
<script type="text/javascript" src="menu.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
$(function() {
  initMenu('',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
/* @license-end */
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="dir_460afd45d0ec637b1427e0e06a6fbcf7.html">net_flow</a></li><li class="navelem"><a class="el" href="dir_6dd0f5e913f2b13fdf6025047ac045ce.html">detail</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle"><div class="title">low_lvl_io.cpp</div></div>
</div><!--header-->
<div class="contents">
<a href="low__lvl__io_8cpp.html">Go to the documentation of this file.</a><div class="fragment"><div class="line"><a id="l00001" name="l00001"></a><span class="lineno">    1</span><span class="comment">/* Flow</span></div>
<div class="line"><a id="l00002" name="l00002"></a><span class="lineno">    2</span><span class="comment"> * Copyright 2023 Akamai Technologies, Inc.</span></div>
<div class="line"><a id="l00003" name="l00003"></a><span class="lineno">    3</span><span class="comment"> *</span></div>
<div class="line"><a id="l00004" name="l00004"></a><span class="lineno">    4</span><span class="comment"> * Licensed under the Apache License, Version 2.0 (the</span></div>
<div class="line"><a id="l00005" name="l00005"></a><span class="lineno">    5</span><span class="comment"> * &quot;License&quot;); you may not use this file except in</span></div>
<div class="line"><a id="l00006" name="l00006"></a><span class="lineno">    6</span><span class="comment"> * compliance with the License.  You may obtain a copy</span></div>
<div class="line"><a id="l00007" name="l00007"></a><span class="lineno">    7</span><span class="comment"> * of the License at</span></div>
<div class="line"><a id="l00008" name="l00008"></a><span class="lineno">    8</span><span class="comment"> *</span></div>
<div class="line"><a id="l00009" name="l00009"></a><span class="lineno">    9</span><span class="comment"> *   https://www.apache.org/licenses/LICENSE-2.0</span></div>
<div class="line"><a id="l00010" name="l00010"></a><span class="lineno">   10</span><span class="comment"> *</span></div>
<div class="line"><a id="l00011" name="l00011"></a><span class="lineno">   11</span><span class="comment"> * Unless required by applicable law or agreed to in</span></div>
<div class="line"><a id="l00012" name="l00012"></a><span class="lineno">   12</span><span class="comment"> * writing, software distributed under the License is</span></div>
<div class="line"><a id="l00013" name="l00013"></a><span class="lineno">   13</span><span class="comment"> * distributed on an &quot;AS IS&quot; BASIS, WITHOUT WARRANTIES OR</span></div>
<div class="line"><a id="l00014" name="l00014"></a><span class="lineno">   14</span><span class="comment"> * CONDITIONS OF ANY KIND, either express or implied.</span></div>
<div class="line"><a id="l00015" name="l00015"></a><span class="lineno">   15</span><span class="comment"> * See the License for the specific language governing</span></div>
<div class="line"><a id="l00016" name="l00016"></a><span class="lineno">   16</span><span class="comment"> * permissions and limitations under the License. */</span></div>
<div class="line"><a id="l00017" name="l00017"></a><span class="lineno">   17</span><span class="comment"></span> </div>
<div class="line"><a id="l00018" name="l00018"></a><span class="lineno">   18</span><span class="comment">/// @file</span></div>
<div class="line"><a id="l00019" name="l00019"></a><span class="lineno">   19</span><span class="comment"></span><span class="preprocessor">#include &quot;<a class="code" href="node_8hpp.html">flow/net_flow/node.hpp</a>&quot;</span></div>
<div class="line"><a id="l00020" name="l00020"></a><span class="lineno">   20</span><span class="preprocessor">#include &quot;<a class="code" href="drop__timer_8hpp.html">flow/net_flow/detail/drop_timer.hpp</a>&quot;</span></div>
<div class="line"><a id="l00021" name="l00021"></a><span class="lineno">   21</span><span class="preprocessor">#include &quot;<a class="code" href="low__lvl__io_8hpp.html">flow/net_flow/detail/low_lvl_io.hpp</a>&quot;</span></div>
<div class="line"><a id="l00022" name="l00022"></a><span class="lineno">   22</span><span class="preprocessor">#include &quot;<a class="code" href="cong__ctl_8hpp.html">flow/net_flow/detail/cong_ctl.hpp</a>&quot;</span></div>
<div class="line"><a id="l00023" name="l00023"></a><span class="lineno">   23</span><span class="preprocessor">#include &quot;<a class="code" href="sched__task_8hpp.html">flow/util/sched_task.hpp</a>&quot;</span></div>
<div class="line"><a id="l00024" name="l00024"></a><span class="lineno">   24</span><span class="preprocessor">#include &lt;utility&gt;</span></div>
<div class="line"><a id="l00025" name="l00025"></a><span class="lineno">   25</span> </div>
<div class="line"><a id="l00026" name="l00026"></a><span class="lineno">   26</span><span class="keyword">namespace </span><a class="code hl_namespace" href="namespaceflow_1_1net__flow.html">flow::net_flow</a></div>
<div class="line"><a id="l00027" name="l00027"></a><span class="lineno">   27</span>{</div>
<div class="line"><a id="l00028" name="l00028"></a><span class="lineno">   28</span> </div>
<div class="line"><a id="l00029" name="l00029"></a><span class="lineno">   29</span><span class="comment">// Implementations.</span></div>
<div class="line"><a id="l00030" name="l00030"></a><span class="lineno">   30</span> </div>
<div class="line"><a id="l00031" name="l00031"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#ade12d2ebbd88b62432089a3a071b9514">   31</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#ade12d2ebbd88b62432089a3a071b9514">Node::async_low_lvl_recv</a>()</div>
<div class="line"><a id="l00032" name="l00032"></a><span class="lineno">   32</span>{</div>
<div class="line"><a id="l00033" name="l00033"></a><span class="lineno">   33</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00034" name="l00034"></a><span class="lineno">   34</span>  <a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#a9b9e0224c17190e99debdfd81640bf89">m_low_lvl_sock</a>.async_wait(Udp_socket::wait_read, [<span class="keyword">this</span>](<span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow.html#a29eaaa9d0fac4ce87d8b969222dbed09">Error_code</a>&amp; sys_err_code)</div>
<div class="line"><a id="l00035" name="l00035"></a><span class="lineno">   35</span>  {</div>
<div class="line"><a id="l00036" name="l00036"></a><span class="lineno">   36</span>    <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a9909cd2b2a77c57da9e35fcef0f8a4e5">low_lvl_recv_and_handle</a>(sys_err_code);</div>
<div class="line"><a id="l00037" name="l00037"></a><span class="lineno">   37</span>  });</div>
<div class="line"><a id="l00038" name="l00038"></a><span class="lineno">   38</span>}</div>
<div class="line"><a id="l00039" name="l00039"></a><span class="lineno">   39</span> </div>
<div class="line"><a id="l00040" name="l00040"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#a9909cd2b2a77c57da9e35fcef0f8a4e5">   40</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a9909cd2b2a77c57da9e35fcef0f8a4e5">Node::low_lvl_recv_and_handle</a>(<a class="code hl_typedef" href="namespaceflow.html#a29eaaa9d0fac4ce87d8b969222dbed09">Error_code</a> sys_err_code)</div>
<div class="line"><a id="l00041" name="l00041"></a><span class="lineno">   41</span>{</div>
<div class="line"><a id="l00042" name="l00042"></a><span class="lineno">   42</span>  <span class="keyword">using </span><a class="code hl_typedef" href="namespaceflow_1_1util.html#af09888b3a8feaff209b6625e493bc338">util::Blob</a>;</div>
<div class="line"><a id="l00043" name="l00043"></a><span class="lineno">   43</span>  <span class="keyword">using </span>boost::asio::buffer;</div>
<div class="line"><a id="l00044" name="l00044"></a><span class="lineno">   44</span> </div>
<div class="line"><a id="l00045" name="l00045"></a><span class="lineno">   45</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00046" name="l00046"></a><span class="lineno">   46</span> </div>
<div class="line"><a id="l00047" name="l00047"></a><span class="lineno">   47</span>  <span class="comment">// Number of packets received and thus handled by handle_incoming().  Useful at least for a log message at the end.</span></div>
<div class="line"><a id="l00048" name="l00048"></a><span class="lineno">   48</span>  <span class="keywordtype">unsigned</span> <span class="keywordtype">int</span> handled_packet_count = 0;</div>
<div class="line"><a id="l00049" name="l00049"></a><span class="lineno">   49</span>  <span class="comment">/* Number of packets received.  (The number handled by handle_incoming() may be lower or higher</span></div>
<div class="line"><a id="l00050" name="l00050"></a><span class="lineno">   50</span><span class="comment">   * if simulator is in use. */</span></div>
<div class="line"><a id="l00051" name="l00051"></a><span class="lineno">   51</span>  <span class="keywordtype">unsigned</span> <span class="keywordtype">int</span> recvd_packet_count = 0;</div>
<div class="line"><a id="l00052" name="l00052"></a><span class="lineno">   52</span> </div>
<div class="line"><a id="l00053" name="l00053"></a><span class="lineno">   53</span>  <span class="comment">/* The limit on the # of received packets to handle in this handler.  0 means unlimited. For</span></div>
<div class="line"><a id="l00054" name="l00054"></a><span class="lineno">   54</span><span class="comment">   * reasoning as to why we&#39;d possibly want to limit it, see doc header for this option. */</span></div>
<div class="line"><a id="l00055" name="l00055"></a><span class="lineno">   55</span>  <span class="keywordtype">unsigned</span> <span class="keywordtype">int</span> recvd_packet_count_limit = <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a43c42121d48a55a43a48e3a84bd10595">opt</a>(<a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#ade08f0457943f803c23b55649139c0d3">m_opts</a>.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Node__options.html#a99cb19cf77ac787103ac616f4415eaf9">m_dyn_max_packets_per_main_loop_iteration</a>);</div>
<div class="line"><a id="l00056" name="l00056"></a><span class="lineno">   56</span> </div>
<div class="line"><a id="l00057" name="l00057"></a><span class="lineno">   57</span>  <span class="keywordflow">if</span> (!sys_err_code)</div>
<div class="line"><a id="l00058" name="l00058"></a><span class="lineno">   58</span>  {</div>
<div class="line"><a id="l00059" name="l00059"></a><span class="lineno">   59</span>    <span class="comment">/* boost.asio has reason to believe there&#39;s at least one UDP datagram ready to read from the</span></div>
<div class="line"><a id="l00060" name="l00060"></a><span class="lineno">   60</span><span class="comment">     * UDP net-stack&#39;s buffer.  We&#39;ll read that one.  We&#39;ll also keep reading them until UDP net-stack</span></div>
<div class="line"><a id="l00061" name="l00061"></a><span class="lineno">   61</span><span class="comment">     * says it &quot;would block&quot; (i.e., no more datagrams have been received).  Why not simply call</span></div>
<div class="line"><a id="l00062" name="l00062"></a><span class="lineno">   62</span><span class="comment">     * async_low_lvl_recv() and let boost.asio deal with it and call us again?  Consider a user</span></div>
<div class="line"><a id="l00063" name="l00063"></a><span class="lineno">   63</span><span class="comment">     * waiting for Readable for a given Peer_socket.   If did that, we will</span></div>
<div class="line"><a id="l00064" name="l00064"></a><span class="lineno">   64</span><span class="comment">     * read one DATA packet, load it on the Receive buffer, and signal the waiting user.  Thus they</span></div>
<div class="line"><a id="l00065" name="l00065"></a><span class="lineno">   65</span><span class="comment">     * may immediately read from the Receive buffer and move onto the rest of their event loop</span></div>
<div class="line"><a id="l00066" name="l00066"></a><span class="lineno">   66</span><span class="comment">     * iteration.  However, there may be 50 more packets we&#39;re then going to immediately put on the</span></div>
<div class="line"><a id="l00067" name="l00067"></a><span class="lineno">   67</span><span class="comment">     * Receive buffer in thread W.  It would be arguably more efficient to read all 51 and THEN</span></div>
<div class="line"><a id="l00068" name="l00068"></a><span class="lineno">   68</span><span class="comment">     * signal the user.  Therefore we use Node::m_sock_events to accumulate active events</span></div>
<div class="line"><a id="l00069" name="l00069"></a><span class="lineno">   69</span><span class="comment">     * and then read all available datagrams; only after that do we then signal Event_sets for</span></div>
<div class="line"><a id="l00070" name="l00070"></a><span class="lineno">   70</span><span class="comment">     * accumulated events.</span></div>
<div class="line"><a id="l00071" name="l00071"></a><span class="lineno">   71</span><span class="comment">     *</span></div>
<div class="line"><a id="l00072" name="l00072"></a><span class="lineno">   72</span><span class="comment">     * @todo One can conceive of some pathological case where due to extreme traffic we&#39;d keep</span></div>
<div class="line"><a id="l00073" name="l00073"></a><span class="lineno">   73</span><span class="comment">     * reading more and more datagrams and not get out of the loop for a long time.  Perhaps add a</span></div>
<div class="line"><a id="l00074" name="l00074"></a><span class="lineno">   74</span><span class="comment">     * knob for the maximum number of iterations to go through before ending the loop and</span></div>
<div class="line"><a id="l00075" name="l00075"></a><span class="lineno">   75</span><span class="comment">     * signaling about the accumulated events. */</span></div>
<div class="line"><a id="l00076" name="l00076"></a><span class="lineno">   76</span> </div>
<div class="line"><a id="l00077" name="l00077"></a><span class="lineno">   77</span>    <span class="comment">// Note: m_packet_data is a member that is reused repeatedly for some performance savings.</span></div>
<div class="line"><a id="l00078" name="l00078"></a><span class="lineno">   78</span>    <span class="keyword">auto</span>&amp; packet_data = <a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#acea4db94c4fb86ddaa2502c2d7e128ea">m_packet_data</a>;</div>
<div class="line"><a id="l00079" name="l00079"></a><span class="lineno">   79</span>    <span class="comment">/* Don&#39;t let this dynamic option&#39;s value&#39;s change to affect this main loop iteration&#39;s value.</span></div>
<div class="line"><a id="l00080" name="l00080"></a><span class="lineno">   80</span><span class="comment">     * This way packet_data.capacity() will essentially remain constant which is easier to reason about than</span></div>
<div class="line"><a id="l00081" name="l00081"></a><span class="lineno">   81</span><span class="comment">     * the alternative.  Soon enough this method will exit, and any new value will take effect next time. */</span></div>
<div class="line"><a id="l00082" name="l00082"></a><span class="lineno">   82</span>    <span class="keyword">const</span> <span class="keywordtype">size_t</span> max_packet_size = <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a43c42121d48a55a43a48e3a84bd10595">opt</a>(<a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#ade08f0457943f803c23b55649139c0d3">m_opts</a>.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Node__options.html#a88c732d10914f406cc43a2c20cc800c6">m_dyn_low_lvl_max_packet_size</a>);</div>
<div class="line"><a id="l00083" name="l00083"></a><span class="lineno">   83</span> </div>
<div class="line"><a id="l00084" name="l00084"></a><span class="lineno">   84</span>    <a class="code hl_typedef" href="namespaceflow_1_1util.html#af1b28edb7bad743c1c77f6b982cc3196">util::Udp_endpoint</a> low_lvl_remote_endpoint;</div>
<div class="line"><a id="l00085" name="l00085"></a><span class="lineno">   85</span> </div>
<div class="line"><a id="l00086" name="l00086"></a><span class="lineno">   86</span>    <span class="comment">// Read until error or &quot;would block&quot; (no data available).</span></div>
<div class="line"><a id="l00087" name="l00087"></a><span class="lineno">   87</span>    <span class="keywordtype">size_t</span> packet_size = 0;</div>
<div class="line"><a id="l00088" name="l00088"></a><span class="lineno">   88</span>    <span class="keywordflow">do</span></div>
<div class="line"><a id="l00089" name="l00089"></a><span class="lineno">   89</span>    {</div>
<div class="line"><a id="l00090" name="l00090"></a><span class="lineno">   90</span>      <span class="comment">/* Make buffer big enough for any packet we&#39;d accept as valid.</span></div>
<div class="line"><a id="l00091" name="l00091"></a><span class="lineno">   91</span><span class="comment">       * resize(MAX) will only work if packet_data.zero() (the case in first iteration) or if packet_data.capacity()</span></div>
<div class="line"><a id="l00092" name="l00092"></a><span class="lineno">   92</span><span class="comment">       * &gt;= MAX.  Hence, if a loop iteration below has left packet_data with unsatisfactory capacity below MAX,</span></div>
<div class="line"><a id="l00093" name="l00093"></a><span class="lineno">   93</span><span class="comment">       * then ensure zero() is true before the resize(MAX) call.  In other words, reallocate packet_data but only</span></div>
<div class="line"><a id="l00094" name="l00094"></a><span class="lineno">   94</span><span class="comment">       * if necessary. */</span></div>
<div class="line"><a id="l00095" name="l00095"></a><span class="lineno">   95</span>      <span class="keywordflow">if</span> ((!packet_data.zero()) &amp;&amp; (packet_data.capacity() &lt; max_packet_size))</div>
<div class="line"><a id="l00096" name="l00096"></a><span class="lineno">   96</span>      {</div>
<div class="line"><a id="l00097" name="l00097"></a><span class="lineno">   97</span>        packet_data.make_zero(); <span class="comment">// This must be done explicitly: acknowledges we force reallocation in next line.</span></div>
<div class="line"><a id="l00098" name="l00098"></a><span class="lineno">   98</span>      }</div>
<div class="line"><a id="l00099" name="l00099"></a><span class="lineno">   99</span>      packet_data.resize(max_packet_size);</div>
<div class="line"><a id="l00100" name="l00100"></a><span class="lineno">  100</span>      <span class="comment">// ^-- Could use UDP available(), but its meaning is slightly ambiguous, + I don&#39;t care for the tiny race it adds.</span></div>
<div class="line"><a id="l00101" name="l00101"></a><span class="lineno">  101</span> </div>
<div class="line"><a id="l00102" name="l00102"></a><span class="lineno">  102</span>      <span class="comment">// Read packet from UDP net-stack internal buffer into &quot;packet_data.&quot;</span></div>
<div class="line"><a id="l00103" name="l00103"></a><span class="lineno">  103</span>      packet_size = <a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#a9b9e0224c17190e99debdfd81640bf89">m_low_lvl_sock</a>.receive_from(buffer(packet_data.data(), packet_data.size()),</div>
<div class="line"><a id="l00104" name="l00104"></a><span class="lineno">  104</span>                                                low_lvl_remote_endpoint, 0, sys_err_code);</div>
<div class="line"><a id="l00105" name="l00105"></a><span class="lineno">  105</span>      <span class="keywordflow">if</span> (!sys_err_code)</div>
<div class="line"><a id="l00106" name="l00106"></a><span class="lineno">  106</span>      {</div>
<div class="line"><a id="l00107" name="l00107"></a><span class="lineno">  107</span>        assert(packet_size &lt;= packet_data.size());</div>
<div class="line"><a id="l00108" name="l00108"></a><span class="lineno">  108</span>        packet_data.resize(packet_size); <span class="comment">// boost.asio NEVER resizes vector to the # of bytes it read.</span></div>
<div class="line"><a id="l00109" name="l00109"></a><span class="lineno">  109</span> </div>
<div class="line"><a id="l00110" name="l00110"></a><span class="lineno">  110</span>        <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Received low-level packet at [UDP &quot;</span> &lt;&lt; <a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#a9b9e0224c17190e99debdfd81640bf89">m_low_lvl_sock</a>.local_endpoint() &lt;&lt; <span class="stringliteral">&quot;] from &quot;</span></div>
<div class="line"><a id="l00111" name="l00111"></a><span class="lineno">  111</span>                       <span class="stringliteral">&quot;[UDP &quot;</span> &lt;&lt; low_lvl_remote_endpoint &lt;&lt; <span class="stringliteral">&quot;]:&quot;</span>);</div>
<div class="line"><a id="l00112" name="l00112"></a><span class="lineno">  112</span> </div>
<div class="line"><a id="l00113" name="l00113"></a><span class="lineno">  113</span>        <span class="comment">// Count it against the limit.</span></div>
<div class="line"><a id="l00114" name="l00114"></a><span class="lineno">  114</span>        ++recvd_packet_count;</div>
<div class="line"><a id="l00115" name="l00115"></a><span class="lineno">  115</span> </div>
<div class="line"><a id="l00116" name="l00116"></a><span class="lineno">  116</span>        handled_packet_count += <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a90ecbad1ff3a8d2fe4b3ad1c474f7513">handle_incoming_with_simulation</a>(&amp;packet_data, low_lvl_remote_endpoint);</div>
<div class="line"><a id="l00117" name="l00117"></a><span class="lineno">  117</span>        <span class="comment">/* packet_data is still valid and owned by us but may have any structure at all.</span></div>
<div class="line"><a id="l00118" name="l00118"></a><span class="lineno">  118</span><span class="comment">         * As of this writing, in practice -- assuming no simulated delay -- packet_data would be untouched</span></div>
<div class="line"><a id="l00119" name="l00119"></a><span class="lineno">  119</span><span class="comment">         * (and therefore could be reused sans reallocation for the next packet read in above, if any) for all</span></div>
<div class="line"><a id="l00120" name="l00120"></a><span class="lineno">  120</span><span class="comment">         * packet types except DATA, which is moved elsewhere via std::move() semantics and would require</span></div>
<div class="line"><a id="l00121" name="l00121"></a><span class="lineno">  121</span><span class="comment">         * reallocation above. */</span></div>
<div class="line"><a id="l00122" name="l00122"></a><span class="lineno">  122</span>      }</div>
<div class="line"><a id="l00123" name="l00123"></a><span class="lineno">  123</span>      <span class="keywordflow">else</span> <span class="keywordflow">if</span> (sys_err_code != boost::asio::error::would_block) <span class="comment">// &quot;Would block&quot; is normal (no data available).</span></div>
<div class="line"><a id="l00124" name="l00124"></a><span class="lineno">  124</span>      {</div>
<div class="line"><a id="l00125" name="l00125"></a><span class="lineno">  125</span>        <span class="comment">/* Weird it failed, since it&#39;s UDP.  Oh well.</span></div>
<div class="line"><a id="l00126" name="l00126"></a><span class="lineno">  126</span><span class="comment">         * Note: One might be tempted to do something dire like close some sockets.  Note that in the current setup</span></div>
<div class="line"><a id="l00127" name="l00127"></a><span class="lineno">  127</span><span class="comment">         * we wouldn&#39;t know which socket(s) to close, since several live on the same low-level (UDP) port.  Even if</span></div>
<div class="line"><a id="l00128" name="l00128"></a><span class="lineno">  128</span><span class="comment">         * we did, the failure to read could mean many things, so individual err_codes would need to be examined</span></div>
<div class="line"><a id="l00129" name="l00129"></a><span class="lineno">  129</span><span class="comment">         * with various actions taken depending on what happened.  (@todo Consider it.)  Without that, just ignore it</span></div>
<div class="line"><a id="l00130" name="l00130"></a><span class="lineno">  130</span><span class="comment">         * as we would any lost packet.  Obviously, we deal with UDP&#39;s unreliability already.</span></div>
<div class="line"><a id="l00131" name="l00131"></a><span class="lineno">  131</span><span class="comment">         *</span></div>
<div class="line"><a id="l00132" name="l00132"></a><span class="lineno">  132</span><span class="comment">         * What about the would_block and try_again (EAGAIN/EWOULDBLOCK in POSIX world and similar for Windows)</span></div>
<div class="line"><a id="l00133" name="l00133"></a><span class="lineno">  133</span><span class="comment">         * error codes which often demand special handling?  Well, firstly, we are likely mostly not hitting that</span></div>
<div class="line"><a id="l00134" name="l00134"></a><span class="lineno">  134</span><span class="comment">         * situation, as async_receive() (the call for which this is the handler function) specifically attempts</span></div>
<div class="line"><a id="l00135" name="l00135"></a><span class="lineno">  135</span><span class="comment">         * to avoid those error codes by only executing the handler once the socket is Readable.</span></div>
<div class="line"><a id="l00136" name="l00136"></a><span class="lineno">  136</span><span class="comment">         * Still, it&#39;s probably not impossible: we used async_wait() as of this writing, which means the actual</span></div>
<div class="line"><a id="l00137" name="l00137"></a><span class="lineno">  137</span><span class="comment">         * receiving is done in this handler, not by boost.asio (and even if it was, we would still try more receives</span></div>
<div class="line"><a id="l00138" name="l00138"></a><span class="lineno">  138</span><span class="comment">         * until no more are available).  Between detection of Readable by boost.asio and the actual receive call,</span></div>
<div class="line"><a id="l00139" name="l00139"></a><span class="lineno">  139</span><span class="comment">         * the situation may have changed.  Well, fine.  What&#39;s there to do?  would_block/try_again means</span></div>
<div class="line"><a id="l00140" name="l00140"></a><span class="lineno">  140</span><span class="comment">         * not Readable right now... try later.  Great!  We do just that in any case below by executing</span></div>
<div class="line"><a id="l00141" name="l00141"></a><span class="lineno">  141</span><span class="comment">         * async_receive() again (inside the call at the end of this method).  So this read failed due to some</span></div>
<div class="line"><a id="l00142" name="l00142"></a><span class="lineno">  142</span><span class="comment">         * odd change in conditions; best we can do is wait for Readable and try again later.  And we do. */</span></div>
<div class="line"><a id="l00143" name="l00143"></a><span class="lineno">  143</span>        <a class="code hl_define" href="error_2error_8hpp.html#a019ee6e90802e444572dd78d312d6e23">FLOW_ERROR_SYS_ERROR_LOG_WARNING</a>();</div>
<div class="line"><a id="l00144" name="l00144"></a><span class="lineno">  144</span>        assert(packet_size == 0); <span class="comment">// Should be the case on error.</span></div>
<div class="line"><a id="l00145" name="l00145"></a><span class="lineno">  145</span>      }</div>
<div class="line"><a id="l00146" name="l00146"></a><span class="lineno">  146</span>      <span class="keywordflow">else</span></div>
<div class="line"><a id="l00147" name="l00147"></a><span class="lineno">  147</span>      {</div>
<div class="line"><a id="l00148" name="l00148"></a><span class="lineno">  148</span>        assert(packet_size == 0);</div>
<div class="line"><a id="l00149" name="l00149"></a><span class="lineno">  149</span>      }</div>
<div class="line"><a id="l00150" name="l00150"></a><span class="lineno">  150</span>    }</div>
<div class="line"><a id="l00151" name="l00151"></a><span class="lineno">  151</span>    <span class="comment">// while (still getting data &amp;&amp; (haven&#39;t exceeded per-handler limit || there is no limit))</span></div>
<div class="line"><a id="l00152" name="l00152"></a><span class="lineno">  152</span>    <span class="keywordflow">while</span> ((packet_size != 0)</div>
<div class="line"><a id="l00153" name="l00153"></a><span class="lineno">  153</span>           &amp;&amp; ((recvd_packet_count_limit == 0) || (recvd_packet_count &lt; recvd_packet_count_limit)));</div>
<div class="line"><a id="l00154" name="l00154"></a><span class="lineno">  154</span> </div>
<div class="line"><a id="l00155" name="l00155"></a><span class="lineno">  155</span>    <span class="comment">/* Have read all available low-level data.  This may have accumulated certain tasks, like</span></div>
<div class="line"><a id="l00156" name="l00156"></a><span class="lineno">  156</span><span class="comment">     * combining pending individual acknowledgments into larger ACK packet(s), to be performed at</span></div>
<div class="line"><a id="l00157" name="l00157"></a><span class="lineno">  157</span><span class="comment">     * the end of the handler.  Do so now. */</span></div>
<div class="line"><a id="l00158" name="l00158"></a><span class="lineno">  158</span>    <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a0fba26db0055a969f3869014bc5c2bfc">perform_accumulated_on_recv_tasks</a>();</div>
<div class="line"><a id="l00159" name="l00159"></a><span class="lineno">  159</span> </div>
<div class="line"><a id="l00160" name="l00160"></a><span class="lineno">  160</span>    <span class="comment">// Helpful in understanding the relationship between handler invocations and &quot;delta&quot; Event_set checks.</span></div>
<div class="line"><a id="l00161" name="l00161"></a><span class="lineno">  161</span>    <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Handled a total of [&quot;</span> &lt;&lt; handled_packet_count &lt;&lt; <span class="stringliteral">&quot;] incoming packets &quot;</span></div>
<div class="line"><a id="l00162" name="l00162"></a><span class="lineno">  162</span>                   <span class="stringliteral">&quot;out of [&quot;</span> &lt;&lt; recvd_packet_count &lt;&lt; <span class="stringliteral">&quot;] received (limit [&quot;</span> &lt;&lt; recvd_packet_count_limit &lt;&lt; <span class="stringliteral">&quot;]) &quot;</span></div>
<div class="line"><a id="l00163" name="l00163"></a><span class="lineno">  163</span>                   <span class="stringliteral">&quot;in this boost.asio handler.&quot;</span>);</div>
<div class="line"><a id="l00164" name="l00164"></a><span class="lineno">  164</span>  } <span class="comment">// if (!err_code)</span></div>
<div class="line"><a id="l00165" name="l00165"></a><span class="lineno">  165</span>  <span class="keywordflow">else</span></div>
<div class="line"><a id="l00166" name="l00166"></a><span class="lineno">  166</span>  {</div>
<div class="line"><a id="l00167" name="l00167"></a><span class="lineno">  167</span>    <span class="comment">// boost.asio called us with an error.  Strange, since this is UDP, but treat it same as a read error above.</span></div>
<div class="line"><a id="l00168" name="l00168"></a><span class="lineno">  168</span>    <a class="code hl_define" href="error_2error_8hpp.html#a019ee6e90802e444572dd78d312d6e23">FLOW_ERROR_SYS_ERROR_LOG_WARNING</a>();</div>
<div class="line"><a id="l00169" name="l00169"></a><span class="lineno">  169</span>  }</div>
<div class="line"><a id="l00170" name="l00170"></a><span class="lineno">  170</span> </div>
<div class="line"><a id="l00171" name="l00171"></a><span class="lineno">  171</span>  <span class="comment">/* If socket errors are detected above, since it&#39;s UDP and connectionless, there is hope we&#39;re</span></div>
<div class="line"><a id="l00172" name="l00172"></a><span class="lineno">  172</span><span class="comment">   * still OK, so still set up wait at the end of this callback.  Do not stop event loop.</span></div>
<div class="line"><a id="l00173" name="l00173"></a><span class="lineno">  173</span><span class="comment">   *</span></div>
<div class="line"><a id="l00174" name="l00174"></a><span class="lineno">  174</span><span class="comment">   * @todo This isn&#39;t necessarily sound.  Could investigate the possible errors and act</span></div>
<div class="line"><a id="l00175" name="l00175"></a><span class="lineno">  175</span><span class="comment">   * accordingly. */</span></div>
<div class="line"><a id="l00176" name="l00176"></a><span class="lineno">  176</span> </div>
<div class="line"><a id="l00177" name="l00177"></a><span class="lineno">  177</span>  <span class="comment">// Register ourselves for the next UDP receive.</span></div>
<div class="line"><a id="l00178" name="l00178"></a><span class="lineno">  178</span>  <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#ade12d2ebbd88b62432089a3a071b9514">async_low_lvl_recv</a>();</div>
<div class="line"><a id="l00179" name="l00179"></a><span class="lineno">  179</span>} <span class="comment">// Node::low_lvl_recv_and_handle()</span></div>
<div class="line"><a id="l00180" name="l00180"></a><span class="lineno">  180</span> </div>
<div class="line"><a id="l00181" name="l00181"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#a90ecbad1ff3a8d2fe4b3ad1c474f7513">  181</a></span><span class="keywordtype">unsigned</span> <span class="keywordtype">int</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a90ecbad1ff3a8d2fe4b3ad1c474f7513">Node::handle_incoming_with_simulation</a>(<a class="code hl_typedef" href="namespaceflow_1_1util.html#af09888b3a8feaff209b6625e493bc338">util::Blob</a>* packet_data,</div>
<div class="line"><a id="l00182" name="l00182"></a><span class="lineno">  182</span>                                                   <span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow_1_1util.html#af1b28edb7bad743c1c77f6b982cc3196">util::Udp_endpoint</a>&amp; low_lvl_remote_endpoint,</div>
<div class="line"><a id="l00183" name="l00183"></a><span class="lineno">  183</span>                                                   <span class="keywordtype">bool</span> is_sim_duplicate_packet)</div>
<div class="line"><a id="l00184" name="l00184"></a><span class="lineno">  184</span>{</div>
<div class="line"><a id="l00185" name="l00185"></a><span class="lineno">  185</span>  <span class="keyword">using </span><a class="code hl_typedef" href="namespaceflow_1_1util.html#af09888b3a8feaff209b6625e493bc338">util::Blob</a>;</div>
<div class="line"><a id="l00186" name="l00186"></a><span class="lineno">  186</span>  <span class="keyword">using </span>boost::chrono::milliseconds; <span class="comment">// Just for the output streaming.</span></div>
<div class="line"><a id="l00187" name="l00187"></a><span class="lineno">  187</span>  <span class="keyword">using </span>boost::chrono::round;</div>
<div class="line"><a id="l00188" name="l00188"></a><span class="lineno">  188</span> </div>
<div class="line"><a id="l00189" name="l00189"></a><span class="lineno">  189</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00190" name="l00190"></a><span class="lineno">  190</span> </div>
<div class="line"><a id="l00191" name="l00191"></a><span class="lineno">  191</span>  <span class="keywordtype">unsigned</span> <span class="keywordtype">int</span> handled = 0; <span class="comment">// How many times we called handle_incoming() inside this invocation.</span></div>
<div class="line"><a id="l00192" name="l00192"></a><span class="lineno">  192</span> </div>
<div class="line"><a id="l00193" name="l00193"></a><span class="lineno">  193</span>  <span class="comment">// Basically we should call handle_incoming(), but we may first have to simulate various network conditions.</span></div>
<div class="line"><a id="l00194" name="l00194"></a><span class="lineno">  194</span> </div>
<div class="line"><a id="l00195" name="l00195"></a><span class="lineno">  195</span>  <span class="keywordflow">if</span> (is_sim_duplicate_packet || (!<a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#a43472c3277cdf5e3f28d4f4f51e3bcc0">m_net_env_sim</a>) || (!<a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#a43472c3277cdf5e3f28d4f4f51e3bcc0">m_net_env_sim</a>-&gt;should_drop_received_packet()))</div>
<div class="line"><a id="l00196" name="l00196"></a><span class="lineno">  196</span>  {</div>
<div class="line"><a id="l00197" name="l00197"></a><span class="lineno">  197</span>    <span class="comment">// See below.</span></div>
<div class="line"><a id="l00198" name="l00198"></a><span class="lineno">  198</span>    <span class="keyword">const</span> <span class="keywordtype">bool</span> must_dupe</div>
<div class="line"><a id="l00199" name="l00199"></a><span class="lineno">  199</span>      = (!is_sim_duplicate_packet) &amp;&amp; <a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#a43472c3277cdf5e3f28d4f4f51e3bcc0">m_net_env_sim</a> &amp;&amp; <a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#a43472c3277cdf5e3f28d4f4f51e3bcc0">m_net_env_sim</a>-&gt;should_duplicate_received_packet();</div>
<div class="line"><a id="l00200" name="l00200"></a><span class="lineno">  200</span> </div>
<div class="line"><a id="l00201" name="l00201"></a><span class="lineno">  201</span>    <a class="code hl_typedef" href="namespaceflow_1_1util.html#af09888b3a8feaff209b6625e493bc338">Blob</a> packet_data_copy(<a class="code hl_function" href="classflow_1_1log_1_1Log__context.html#af4db395042b065b00398cd59845dcb4d">get_logger</a>());</div>
<div class="line"><a id="l00202" name="l00202"></a><span class="lineno">  202</span>    <span class="keywordflow">if</span> (must_dupe)</div>
<div class="line"><a id="l00203" name="l00203"></a><span class="lineno">  203</span>    {</div>
<div class="line"><a id="l00204" name="l00204"></a><span class="lineno">  204</span>      <span class="comment">/* We will simulate duplication of the packet below.  Since packet handling can be</span></div>
<div class="line"><a id="l00205" name="l00205"></a><span class="lineno">  205</span><span class="comment">       * destructive (for performance reasons -- see handle_data_to_established()), we must create</span></div>
<div class="line"><a id="l00206" name="l00206"></a><span class="lineno">  206</span><span class="comment">       * a copy that will not be hurt by this process. */</span></div>
<div class="line"><a id="l00207" name="l00207"></a><span class="lineno">  207</span>      packet_data_copy = *(<span class="keyword">static_cast&lt;</span><span class="keyword">const </span><a class="code hl_typedef" href="namespaceflow_1_1util.html#af09888b3a8feaff209b6625e493bc338">Blob</a>*<span class="keyword">&gt;</span>(packet_data)); <span class="comment">// Add const to express we require a copy, not move.</span></div>
<div class="line"><a id="l00208" name="l00208"></a><span class="lineno">  208</span>    }</div>
<div class="line"><a id="l00209" name="l00209"></a><span class="lineno">  209</span> </div>
<div class="line"><a id="l00210" name="l00210"></a><span class="lineno">  210</span>    <a class="code hl_typedef" href="namespaceflow.html#a48799f1263cdeedec125be51a3db2b79">Fine_duration</a> latency(<a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#a43472c3277cdf5e3f28d4f4f51e3bcc0">m_net_env_sim</a> ? <a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#a43472c3277cdf5e3f28d4f4f51e3bcc0">m_net_env_sim</a>-&gt;received_packet_latency() : Fine_duration::zero());</div>
<div class="line"><a id="l00211" name="l00211"></a><span class="lineno">  211</span>    <span class="keywordflow">if</span> (latency == Fine_duration::zero())</div>
<div class="line"><a id="l00212" name="l00212"></a><span class="lineno">  212</span>    {</div>
<div class="line"><a id="l00213" name="l00213"></a><span class="lineno">  213</span>      <span class="comment">// No simulated latency; just handle the packet now (mainstream case).</span></div>
<div class="line"><a id="l00214" name="l00214"></a><span class="lineno">  214</span>      <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a1699dfdaf3e5ebe1b1d715a9c306d116">handle_incoming</a>(packet_data, low_lvl_remote_endpoint);</div>
<div class="line"><a id="l00215" name="l00215"></a><span class="lineno">  215</span>      <span class="comment">// *packet_data may now be decimated!  Do not use it.</span></div>
<div class="line"><a id="l00216" name="l00216"></a><span class="lineno">  216</span>      ++handled;</div>
<div class="line"><a id="l00217" name="l00217"></a><span class="lineno">  217</span>    }</div>
<div class="line"><a id="l00218" name="l00218"></a><span class="lineno">  218</span>    <span class="keywordflow">else</span></div>
<div class="line"><a id="l00219" name="l00219"></a><span class="lineno">  219</span>    {</div>
<div class="line"><a id="l00220" name="l00220"></a><span class="lineno">  220</span>      <span class="comment">/* Pretend it was actually delivered later, as the simulator told us to do.  More precisely,</span></div>
<div class="line"><a id="l00221" name="l00221"></a><span class="lineno">  221</span><span class="comment">       * call handle_incoming() but later. */</span></div>
<div class="line"><a id="l00222" name="l00222"></a><span class="lineno">  222</span>      <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;SIMULATION: Delaying reception of packet by simulated latency &quot;</span></div>
<div class="line"><a id="l00223" name="l00223"></a><span class="lineno">  223</span>                     <span class="stringliteral">&quot;[&quot;</span> &lt;&lt; round&lt;milliseconds&gt;(latency) &lt;&lt; <span class="stringliteral">&quot;].&quot;</span>);</div>
<div class="line"><a id="l00224" name="l00224"></a><span class="lineno">  224</span>      <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a4368247d07e930bdd07ab8d2425c6de8">async_wait_latency_then_handle_incoming</a>(latency, packet_data, low_lvl_remote_endpoint);</div>
<div class="line"><a id="l00225" name="l00225"></a><span class="lineno">  225</span>      <span class="comment">// *packet_data may now be decimated!  Do not use it.</span></div>
<div class="line"><a id="l00226" name="l00226"></a><span class="lineno">  226</span>    }</div>
<div class="line"><a id="l00227" name="l00227"></a><span class="lineno">  227</span> </div>
<div class="line"><a id="l00228" name="l00228"></a><span class="lineno">  228</span>    <span class="comment">// Possibly simulate packet duplication.</span></div>
<div class="line"><a id="l00229" name="l00229"></a><span class="lineno">  229</span>    <span class="keywordflow">if</span> (must_dupe)</div>
<div class="line"><a id="l00230" name="l00230"></a><span class="lineno">  230</span>    {</div>
<div class="line"><a id="l00231" name="l00231"></a><span class="lineno">  231</span>      <span class="comment">/* Simulator told us to pretend this packet was received twice.  Note that we only model a</span></div>
<div class="line"><a id="l00232" name="l00232"></a><span class="lineno">  232</span><span class="comment">       * single duplication (i.e., a duplicated packet will not be simulated to itself get</span></div>
<div class="line"><a id="l00233" name="l00233"></a><span class="lineno">  233</span><span class="comment">       * duplicated).  Also, we don&#39;t drop a duplicated packet. */</span></div>
<div class="line"><a id="l00234" name="l00234"></a><span class="lineno">  234</span>      <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;SIMULATION: Duplicating received packet.&quot;</span>);</div>
<div class="line"><a id="l00235" name="l00235"></a><span class="lineno">  235</span>      handled += <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a90ecbad1ff3a8d2fe4b3ad1c474f7513">handle_incoming_with_simulation</a>(&amp;packet_data_copy, <span class="comment">// Pass the COPY, not the possibly damaged original.</span></div>
<div class="line"><a id="l00236" name="l00236"></a><span class="lineno">  236</span>                                                 low_lvl_remote_endpoint, <span class="keyword">true</span>);</div>
<div class="line"><a id="l00237" name="l00237"></a><span class="lineno">  237</span>      <span class="comment">// packet_data_copy may now be decimated!  Do not use it.</span></div>
<div class="line"><a id="l00238" name="l00238"></a><span class="lineno">  238</span>    }</div>
<div class="line"><a id="l00239" name="l00239"></a><span class="lineno">  239</span>  }</div>
<div class="line"><a id="l00240" name="l00240"></a><span class="lineno">  240</span>  <span class="keywordflow">else</span></div>
<div class="line"><a id="l00241" name="l00241"></a><span class="lineno">  241</span>  {</div>
<div class="line"><a id="l00242" name="l00242"></a><span class="lineno">  242</span>    <span class="comment">// Got packet, but pretend it was dropped on the way due to packet loss, as simulator told us to do.</span></div>
<div class="line"><a id="l00243" name="l00243"></a><span class="lineno">  243</span>    <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;SIMULATION: Dropped received packet.&quot;</span>);</div>
<div class="line"><a id="l00244" name="l00244"></a><span class="lineno">  244</span>  }</div>
<div class="line"><a id="l00245" name="l00245"></a><span class="lineno">  245</span> </div>
<div class="line"><a id="l00246" name="l00246"></a><span class="lineno">  246</span>  <span class="keywordflow">return</span> handled;</div>
<div class="line"><a id="l00247" name="l00247"></a><span class="lineno">  247</span>} <span class="comment">// Node::handle_incoming_with_simulation()</span></div>
<div class="line"><a id="l00248" name="l00248"></a><span class="lineno">  248</span> </div>
<div class="line"><a id="l00249" name="l00249"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#a4368247d07e930bdd07ab8d2425c6de8">  249</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a4368247d07e930bdd07ab8d2425c6de8">Node::async_wait_latency_then_handle_incoming</a>(<span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow.html#a48799f1263cdeedec125be51a3db2b79">Fine_duration</a>&amp; latency,</div>
<div class="line"><a id="l00250" name="l00250"></a><span class="lineno">  250</span>                                                   <a class="code hl_typedef" href="namespaceflow_1_1util.html#af09888b3a8feaff209b6625e493bc338">util::Blob</a>* packet_data,</div>
<div class="line"><a id="l00251" name="l00251"></a><span class="lineno">  251</span>                                                   <span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow_1_1util.html#af1b28edb7bad743c1c77f6b982cc3196">util::Udp_endpoint</a>&amp; low_lvl_remote_endpoint)</div>
<div class="line"><a id="l00252" name="l00252"></a><span class="lineno">  252</span>{</div>
<div class="line"><a id="l00253" name="l00253"></a><span class="lineno">  253</span>  <span class="keyword">using </span><a class="code hl_typedef" href="namespaceflow_1_1util.html#af09888b3a8feaff209b6625e493bc338">util::Blob</a>;</div>
<div class="line"><a id="l00254" name="l00254"></a><span class="lineno">  254</span>  <span class="keyword">using </span><a class="code hl_function" href="namespaceflow_1_1util.html#a74d4a66fb00055be4e1bd6ee98e04667">util::schedule_task_from_now</a>;</div>
<div class="line"><a id="l00255" name="l00255"></a><span class="lineno">  255</span>  <span class="keyword">using </span>boost::chrono::milliseconds;</div>
<div class="line"><a id="l00256" name="l00256"></a><span class="lineno">  256</span>  <span class="keyword">using </span>boost::chrono::round;</div>
<div class="line"><a id="l00257" name="l00257"></a><span class="lineno">  257</span>  <span class="keyword">using </span>boost::shared_ptr;</div>
<div class="line"><a id="l00258" name="l00258"></a><span class="lineno">  258</span> </div>
<div class="line"><a id="l00259" name="l00259"></a><span class="lineno">  259</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00260" name="l00260"></a><span class="lineno">  260</span> </div>
<div class="line"><a id="l00261" name="l00261"></a><span class="lineno">  261</span>  <span class="comment">// Schedule call to handle_incoming() to occur asynchronously after the latency period passes.</span></div>
<div class="line"><a id="l00262" name="l00262"></a><span class="lineno">  262</span> </div>
<div class="line"><a id="l00263" name="l00263"></a><span class="lineno">  263</span>  <span class="comment">/* As advertised, *packet_data loses its buffer into this new container, so that caller can immediately</span></div>
<div class="line"><a id="l00264" name="l00264"></a><span class="lineno">  264</span><span class="comment">   * use it for whatever they want.  Meanwhile, we asynchronously own the actual data in it now.</span></div>
<div class="line"><a id="l00265" name="l00265"></a><span class="lineno">  265</span><span class="comment">   * Make a smart pointer to ensure it lives long enough for handler to execute... but likely no longer than that. */</span></div>
<div class="line"><a id="l00266" name="l00266"></a><span class="lineno">  266</span>  shared_ptr&lt;Blob&gt; packet_data_moved_ptr(<span class="keyword">new</span> <a class="code hl_typedef" href="namespaceflow_1_1util.html#af09888b3a8feaff209b6625e493bc338">Blob</a>(std::move(*packet_data)));</div>
<div class="line"><a id="l00267" name="l00267"></a><span class="lineno">  267</span> </div>
<div class="line"><a id="l00268" name="l00268"></a><span class="lineno">  268</span>  <span class="comment">// Unused if it doesn&#39;t get logged, which is a slight perf hit, but anyway this sim feature is a debug/test thing.</span></div>
<div class="line"><a id="l00269" name="l00269"></a><span class="lineno">  269</span>  <span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow.html#a9d9cc2eeb10d398cff5591d446b763b8">Fine_time_pt</a> started_at = Fine_clock::now();</div>
<div class="line"><a id="l00270" name="l00270"></a><span class="lineno">  270</span> </div>
<div class="line"><a id="l00271" name="l00271"></a><span class="lineno">  271</span>  <a class="code hl_function" href="namespaceflow_1_1util.html#a74d4a66fb00055be4e1bd6ee98e04667">schedule_task_from_now</a>(<a class="code hl_function" href="classflow_1_1log_1_1Log__context.html#af4db395042b065b00398cd59845dcb4d">get_logger</a>(), latency, <span class="keyword">true</span>, &amp;<a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#af5597a417d03e2bed87cd772ef5cef29">m_task_engine</a>,</div>
<div class="line"><a id="l00272" name="l00272"></a><span class="lineno">  272</span>                         [<span class="keyword">this</span>, packet_data_moved_ptr, low_lvl_remote_endpoint, latency, started_at]</div>
<div class="line"><a id="l00273" name="l00273"></a><span class="lineno">  273</span>                           (<span class="keywordtype">bool</span>)</div>
<div class="line"><a id="l00274" name="l00274"></a><span class="lineno">  274</span>  {</div>
<div class="line"><a id="l00275" name="l00275"></a><span class="lineno">  275</span>    <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00276" name="l00276"></a><span class="lineno">  276</span> </div>
<div class="line"><a id="l00277" name="l00277"></a><span class="lineno">  277</span>    <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a></div>
<div class="line"><a id="l00278" name="l00278"></a><span class="lineno">  278</span>      (<span class="stringliteral">&quot;SIMULATOR: Handling low-level packet after &quot;</span></div>
<div class="line"><a id="l00279" name="l00279"></a><span class="lineno">  279</span>       <span class="stringliteral">&quot;simulated latency [&quot;</span> &lt;&lt; round&lt;milliseconds&gt;(latency) &lt;&lt; <span class="stringliteral">&quot;]; &quot;</span></div>
<div class="line"><a id="l00280" name="l00280"></a><span class="lineno">  280</span>       <span class="stringliteral">&quot;actual simulated latency was &quot;</span></div>
<div class="line"><a id="l00281" name="l00281"></a><span class="lineno">  281</span>       <span class="stringliteral">&quot;[&quot;</span> &lt;&lt; round&lt;milliseconds&gt;(Fine_clock::now() - started_at) &lt;&lt; <span class="stringliteral">&quot;]; &quot;</span></div>
<div class="line"><a id="l00282" name="l00282"></a><span class="lineno">  282</span>       <span class="stringliteral">&quot;from [UDP &quot;</span> &lt;&lt; low_lvl_remote_endpoint &lt;&lt; <span class="stringliteral">&quot;].&quot;</span>);</div>
<div class="line"><a id="l00283" name="l00283"></a><span class="lineno">  283</span> </div>
<div class="line"><a id="l00284" name="l00284"></a><span class="lineno">  284</span>    <span class="comment">// Move (again) the actual buffer to handle_incoming()&#39;s ownership.</span></div>
<div class="line"><a id="l00285" name="l00285"></a><span class="lineno">  285</span>    <a class="code hl_typedef" href="namespaceflow_1_1util.html#af09888b3a8feaff209b6625e493bc338">Blob</a> packet_data_moved_again(std::move(*packet_data_moved_ptr));</div>
<div class="line"><a id="l00286" name="l00286"></a><span class="lineno">  286</span>    <span class="comment">// *packet_data_moved_ptr is now empty and will be deleted once that smart pointer goes out of scope below.</span></div>
<div class="line"><a id="l00287" name="l00287"></a><span class="lineno">  287</span>    <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a1699dfdaf3e5ebe1b1d715a9c306d116">handle_incoming</a>(&amp;packet_data_moved_again, low_lvl_remote_endpoint);</div>
<div class="line"><a id="l00288" name="l00288"></a><span class="lineno">  288</span>    <span class="comment">// packet_data_moved_again may now be decimated also!  Do not use it.</span></div>
<div class="line"><a id="l00289" name="l00289"></a><span class="lineno">  289</span> </div>
<div class="line"><a id="l00290" name="l00290"></a><span class="lineno">  290</span>    <span class="comment">/* We must do this here for similar reasons as at the end of low_lvl_recv_and_handle().  Think</span></div>
<div class="line"><a id="l00291" name="l00291"></a><span class="lineno">  291</span><span class="comment">     * of the present closure as simply low_lvl_recv_and_handle() running and being able</span></div>
<div class="line"><a id="l00292" name="l00292"></a><span class="lineno">  292</span><span class="comment">     * to read off the one low-level UDP packet (the argument &quot;packet&quot;). */</span></div>
<div class="line"><a id="l00293" name="l00293"></a><span class="lineno">  293</span>    <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a0fba26db0055a969f3869014bc5c2bfc">perform_accumulated_on_recv_tasks</a>();</div>
<div class="line"><a id="l00294" name="l00294"></a><span class="lineno">  294</span> </div>
<div class="line"><a id="l00295" name="l00295"></a><span class="lineno">  295</span>    <span class="comment">// Log a similar thing to that in low_lvl_recv_and_handle().</span></div>
<div class="line"><a id="l00296" name="l00296"></a><span class="lineno">  296</span>    <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Handled a total of [1] incoming packets &quot;</span></div>
<div class="line"><a id="l00297" name="l00297"></a><span class="lineno">  297</span>                   <span class="stringliteral">&quot;out of a simulated [1] received in this boost.asio handler.&quot;</span>);</div>
<div class="line"><a id="l00298" name="l00298"></a><span class="lineno">  298</span>  }); <span class="comment">// Async callback.</span></div>
<div class="line"><a id="l00299" name="l00299"></a><span class="lineno">  299</span>} <span class="comment">// Node::async_wait_latency_then_handle_incoming()</span></div>
<div class="line"><a id="l00300" name="l00300"></a><span class="lineno">  300</span> </div>
<div class="line"><a id="l00301" name="l00301"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#a0db2bb587e0407077b11368df075f0ff">  301</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a0db2bb587e0407077b11368df075f0ff">Node::async_sock_low_lvl_packet_send</a>(<a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Peer_socket::Ptr</a> sock, <a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aef7998db71c60eeb5d1e3d1a97c14886">Low_lvl_packet::Const_ptr</a>&amp;&amp; packet,</div>
<div class="line"><a id="l00302" name="l00302"></a><span class="lineno">  302</span>                                          <span class="keywordtype">bool</span> delayed_by_pacing)</div>
<div class="line"><a id="l00303" name="l00303"></a><span class="lineno">  303</span>{</div>
<div class="line"><a id="l00304" name="l00304"></a><span class="lineno">  304</span>  <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a9f302fe63e36f72931e1a759b0aaaed4">async_low_lvl_packet_send_impl</a>(sock-&gt;remote_endpoint().m_udp_endpoint, std::move(packet), delayed_by_pacing, sock);</div>
<div class="line"><a id="l00305" name="l00305"></a><span class="lineno">  305</span>}</div>
<div class="line"><a id="l00306" name="l00306"></a><span class="lineno">  306</span> </div>
<div class="line"><a id="l00307" name="l00307"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#a08a5752bf3848f31abbe753be47c1fcf">  307</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a08a5752bf3848f31abbe753be47c1fcf">Node::async_no_sock_low_lvl_packet_send</a>(<span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow_1_1util.html#af1b28edb7bad743c1c77f6b982cc3196">util::Udp_endpoint</a>&amp; low_lvl_remote_endpoint,</div>
<div class="line"><a id="l00308" name="l00308"></a><span class="lineno">  308</span>                                             <a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aef7998db71c60eeb5d1e3d1a97c14886">Low_lvl_packet::Const_ptr</a> packet)</div>
<div class="line"><a id="l00309" name="l00309"></a><span class="lineno">  309</span>{</div>
<div class="line"><a id="l00310" name="l00310"></a><span class="lineno">  310</span>  <span class="comment">/* As of this writing we don&#39;t pace things, when no Peer_socket is involved (e.g., some RSTs) =&gt; always `false`: -|</span></div>
<div class="line"><a id="l00311" name="l00311"></a><span class="lineno">  311</span><span class="comment">   *                                                              v-------------------------------------------------| */</span></div>
<div class="line"><a id="l00312" name="l00312"></a><span class="lineno">  312</span>  <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a9f302fe63e36f72931e1a759b0aaaed4">async_low_lvl_packet_send_impl</a>(low_lvl_remote_endpoint, packet, <span class="keyword">false</span>, <a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Peer_socket::Ptr</a>());</div>
<div class="line"><a id="l00313" name="l00313"></a><span class="lineno">  313</span>}</div>
<div class="line"><a id="l00314" name="l00314"></a><span class="lineno">  314</span> </div>
<div class="line"><a id="l00315" name="l00315"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#a9f302fe63e36f72931e1a759b0aaaed4">  315</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a9f302fe63e36f72931e1a759b0aaaed4">Node::async_low_lvl_packet_send_impl</a>(<span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow_1_1util.html#af1b28edb7bad743c1c77f6b982cc3196">util::Udp_endpoint</a>&amp; low_lvl_remote_endpoint,</div>
<div class="line"><a id="l00316" name="l00316"></a><span class="lineno">  316</span>                                          <a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aef7998db71c60eeb5d1e3d1a97c14886">Low_lvl_packet::Const_ptr</a> packet,</div>
<div class="line"><a id="l00317" name="l00317"></a><span class="lineno">  317</span>                                          <span class="keywordtype">bool</span> delayed_by_pacing, <a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Peer_socket::Ptr</a> sock)</div>
<div class="line"><a id="l00318" name="l00318"></a><span class="lineno">  318</span>{</div>
<div class="line"><a id="l00319" name="l00319"></a><span class="lineno">  319</span>  <span class="keyword">using </span>boost::asio::buffer;</div>
<div class="line"><a id="l00320" name="l00320"></a><span class="lineno">  320</span> </div>
<div class="line"><a id="l00321" name="l00321"></a><span class="lineno">  321</span>  assert(packet);</div>
<div class="line"><a id="l00322" name="l00322"></a><span class="lineno">  322</span>  <span class="keyword">const</span> <span class="keyword">auto</span>&amp; packet_ref = *packet;</div>
<div class="line"><a id="l00323" name="l00323"></a><span class="lineno">  323</span>  <span class="keyword">const</span> <span class="keyword">auto</span>&amp; packet_type_id = <span class="keyword">typeid</span>(packet_ref);</div>
<div class="line"><a id="l00324" name="l00324"></a><span class="lineno">  324</span> </div>
<div class="line"><a id="l00325" name="l00325"></a><span class="lineno">  325</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00326" name="l00326"></a><span class="lineno">  326</span> </div>
<div class="line"><a id="l00327" name="l00327"></a><span class="lineno">  327</span>  <a class="code hl_class" href="classflow_1_1net__flow_1_1Sequence__number.html">Sequence_number</a> seq_num; <span class="comment">// See below.</span></div>
<div class="line"><a id="l00328" name="l00328"></a><span class="lineno">  328</span> </div>
<div class="line"><a id="l00329" name="l00329"></a><span class="lineno">  329</span>  <span class="comment">/* As explained in send_worker(), if it&#39;s a DATA packet, then we must overwrite</span></div>
<div class="line"><a id="l00330" name="l00330"></a><span class="lineno">  330</span><span class="comment">   * m_sent_when. That way we&#39;ll avoid the deadly RTT drift (increase) due to pacing causing a</span></div>
<div class="line"><a id="l00331" name="l00331"></a><span class="lineno">  331</span><span class="comment">   * higher RTT, then using that RTT to spread out packets more, which causes higher RTT again,</span></div>
<div class="line"><a id="l00332" name="l00332"></a><span class="lineno">  332</span><span class="comment">   * etc.  (If pacing is disabled, then the effect of doing this here rather than in send_worker()</span></div>
<div class="line"><a id="l00333" name="l00333"></a><span class="lineno">  333</span><span class="comment">   * will not be dramatic.)</span></div>
<div class="line"><a id="l00334" name="l00334"></a><span class="lineno">  334</span><span class="comment">   *</span></div>
<div class="line"><a id="l00335" name="l00335"></a><span class="lineno">  335</span><span class="comment">   * Note that a number of important structures are based on m_sent_when; because m_sent_when</span></div>
<div class="line"><a id="l00336" name="l00336"></a><span class="lineno">  336</span><span class="comment">   * ordering determines how things are ordered in m_snd_flying_pkts_by_sent_when and friends and thus the</span></div>
<div class="line"><a id="l00337" name="l00337"></a><span class="lineno">  337</span><span class="comment">   * operation of the Drop Timer.  So whenever we do finalize m_sent_when, we should handle those</span></div>
<div class="line"><a id="l00338" name="l00338"></a><span class="lineno">  338</span><span class="comment">   * things too. */</span></div>
<div class="line"><a id="l00339" name="l00339"></a><span class="lineno">  339</span>  <span class="keywordflow">if</span> (packet_type_id == <span class="keyword">typeid</span>(<a class="code hl_struct" href="structflow_1_1net__flow_1_1Data__packet.html">Data_packet</a>))</div>
<div class="line"><a id="l00340" name="l00340"></a><span class="lineno">  340</span>  {</div>
<div class="line"><a id="l00341" name="l00341"></a><span class="lineno">  341</span>    assert(sock);</div>
<div class="line"><a id="l00342" name="l00342"></a><span class="lineno">  342</span>    <span class="comment">/* OK, so this is where I update m_sent_when and handle everything related to that</span></div>
<div class="line"><a id="l00343" name="l00343"></a><span class="lineno">  343</span><span class="comment">     * (updating the scoreboard, Drop Timer, etc.).  async_send_to() will indeed invoke UDP sendto()</span></div>
<div class="line"><a id="l00344" name="l00344"></a><span class="lineno">  344</span><span class="comment">     * synchronously; that call occurs just below. */</span></div>
<div class="line"><a id="l00345" name="l00345"></a><span class="lineno">  345</span> </div>
<div class="line"><a id="l00346" name="l00346"></a><span class="lineno">  346</span>    <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a2983d0d8a4d4eb5c3402f63f68ef44d8">mark_data_packet_sent</a>(sock, <span class="keyword">static_cast&lt;</span><span class="keyword">const </span><a class="code hl_struct" href="structflow_1_1net__flow_1_1Data__packet.html">Data_packet</a>&amp;<span class="keyword">&gt;</span>(packet_ref).m_seq_num);</div>
<div class="line"><a id="l00347" name="l00347"></a><span class="lineno">  347</span>  } <span class="comment">// if (packet type is DATA)</span></div>
<div class="line"><a id="l00348" name="l00348"></a><span class="lineno">  348</span>  <span class="comment">// @todo See the @todo in Node::async_low_lvl_ack_send() regarding correcting ACK delay value due to pacing.</span></div>
<div class="line"><a id="l00349" name="l00349"></a><span class="lineno">  349</span> </div>
<div class="line"><a id="l00350" name="l00350"></a><span class="lineno">  350</span>  <span class="comment">/* Serialize to raw data.  Important subtleties:</span></div>
<div class="line"><a id="l00351" name="l00351"></a><span class="lineno">  351</span><span class="comment">   *</span></div>
<div class="line"><a id="l00352" name="l00352"></a><span class="lineno">  352</span><span class="comment">   * This does *not* create a copied buffer.  It generates a Const_buffer_sequence,</span></div>
<div class="line"><a id="l00353" name="l00353"></a><span class="lineno">  353</span><span class="comment">   * which is a sequence container (like vector) containing a series of buffer</span></div>
<div class="line"><a id="l00354" name="l00354"></a><span class="lineno">  354</span><span class="comment">   * start pointers and associated lengths.  So it is a &quot;scattered&quot; buffer already in memory, namely within</span></div>
<div class="line"><a id="l00355" name="l00355"></a><span class="lineno">  355</span><span class="comment">   * the Low_lvl_packet `packet`; and the boost.asio UDP async_send_to() we call below performs a &quot;gather&quot;</span></div>
<div class="line"><a id="l00356" name="l00356"></a><span class="lineno">  356</span><span class="comment">   * operation when it generates the UDP datagram to send out.  This should be superior to performing a</span></div>
<div class="line"><a id="l00357" name="l00357"></a><span class="lineno">  357</span><span class="comment">   * copy that is at least proportional (in speed and memory use) to the size of the ultimate datagram.</span></div>
<div class="line"><a id="l00358" name="l00358"></a><span class="lineno">  358</span><span class="comment">   * (In fact, an earlier version of the code indeed serialized to a newly generated single buffer here.)</span></div>
<div class="line"><a id="l00359" name="l00359"></a><span class="lineno">  359</span><span class="comment">   *</span></div>
<div class="line"><a id="l00360" name="l00360"></a><span class="lineno">  360</span><span class="comment">   * The only thing one must be careful of here is that the generated Const_buffer_sequence is valid only as long</span></div>
<div class="line"><a id="l00361" name="l00361"></a><span class="lineno">  361</span><span class="comment">   * as the underlying areas in memory continue to exist; otherwise it&#39;s just a bunch of pointers into invalid</span></div>
<div class="line"><a id="l00362" name="l00362"></a><span class="lineno">  362</span><span class="comment">   * memory.  The underlying memory, as documented in Low_lvl_packet::serialize_to_raw_data() doc header,</span></div>
<div class="line"><a id="l00363" name="l00363"></a><span class="lineno">  363</span><span class="comment">   * is valid as long as the Low_lvl_packet on which the method is called exists.  For this reason, we pass</span></div>
<div class="line"><a id="l00364" name="l00364"></a><span class="lineno">  364</span><span class="comment">   * `packet` (which is a shared_ptr&lt;&gt;) to the completion handler.  Thus it will continue to exist until</span></div>
<div class="line"><a id="l00365" name="l00365"></a><span class="lineno">  365</span><span class="comment">   * after the async_send_to() attempt itself.  (If we hadn&#39;t dobe this, there&#39;s a large danger that</span></div>
<div class="line"><a id="l00366" name="l00366"></a><span class="lineno">  366</span><span class="comment">   * `packet`&#39;s ref-count would drop to zero at method exit just below, and hence async_send_to() asynchronously</span></div>
<div class="line"><a id="l00367" name="l00367"></a><span class="lineno">  367</span><span class="comment">   * would crash a little later.)</span></div>
<div class="line"><a id="l00368" name="l00368"></a><span class="lineno">  368</span><span class="comment">   *</span></div>
<div class="line"><a id="l00369" name="l00369"></a><span class="lineno">  369</span><span class="comment">   * Update: Through empirical evidence, I found that, at least with Boost 1.63 and macOS, UDP async_send_to()</span></div>
<div class="line"><a id="l00370" name="l00370"></a><span class="lineno">  370</span><span class="comment">   * below will silently truncate to the first 64 elements of raw_bufs.  I could find no evidence of this being</span></div>
<div class="line"><a id="l00371" name="l00371"></a><span class="lineno">  371</span><span class="comment">   * a common OS limitation (I&#39;ve found other limits like 1024 in Linux for writev()), though I didn&#39;t dig very hard.</span></div>
<div class="line"><a id="l00372" name="l00372"></a><span class="lineno">  372</span><span class="comment">   * The silent truncation was quite alarming and something to look into if only for educational purposes.</span></div>
<div class="line"><a id="l00373" name="l00373"></a><span class="lineno">  373</span><span class="comment">   * In any case, we must keep raw_bufs.size() fairly low -- probably this is good for performance as well.</span></div>
<div class="line"><a id="l00374" name="l00374"></a><span class="lineno">  374</span><span class="comment">   *</span></div>
<div class="line"><a id="l00375" name="l00375"></a><span class="lineno">  375</span><span class="comment">   * Update: I have since looked into this in another project.  This limit, 64, continues in Boost 1.75 and is actually</span></div>
<div class="line"><a id="l00376" name="l00376"></a><span class="lineno">  376</span><span class="comment">   * at least imposed by boost.asio code itself, though it might just be using that value as a lowest-common-denominator</span></div>
<div class="line"><a id="l00377" name="l00377"></a><span class="lineno">  377</span><span class="comment">   * technique.  It is my impression that they do this silently on account of thinking only of the TCP context,</span></div>
<div class="line"><a id="l00378" name="l00378"></a><span class="lineno">  378</span><span class="comment">   * where a send/write op call is free to write as few bytes as it wants, and one must deal with it in any</span></div>
<div class="line"><a id="l00379" name="l00379"></a><span class="lineno">  379</span><span class="comment">   * case (until would-block).  The silent dropping behavior in UDP, however, is a disaster and is really a Boost bug</span></div>
<div class="line"><a id="l00380" name="l00380"></a><span class="lineno">  380</span><span class="comment">   * (@todo file it against Boost if needed).  In any case it is what it is, and until it goes away, we must not</span></div>
<div class="line"><a id="l00381" name="l00381"></a><span class="lineno">  381</span><span class="comment">   * exceed it. */</span></div>
<div class="line"><a id="l00382" name="l00382"></a><span class="lineno">  382</span>  <a class="code hl_typedef" href="structflow_1_1net__flow_1_1Low__lvl__packet.html#a6e16e9c355c6d5e2acff5f9bdc24f397">Low_lvl_packet::Const_buffer_sequence</a> raw_bufs;</div>
<div class="line"><a id="l00383" name="l00383"></a><span class="lineno">  383</span>  <span class="keyword">const</span> <span class="keywordtype">size_t</span> bytes_to_send = packet-&gt;serialize_to_raw_data_and_log(&amp;raw_bufs);</div>
<div class="line"><a id="l00384" name="l00384"></a><span class="lineno">  384</span>  assert(bytes_to_send != 0);</div>
<div class="line"><a id="l00385" name="l00385"></a><span class="lineno">  385</span> </div>
<div class="line"><a id="l00386" name="l00386"></a><span class="lineno">  386</span>  <span class="comment">// Count an actual UDP stack send() call.</span></div>
<div class="line"><a id="l00387" name="l00387"></a><span class="lineno">  387</span>  sock-&gt;m_snd_stats.low_lvl_packet_xfer_called(packet_type_id, delayed_by_pacing, bytes_to_send);</div>
<div class="line"><a id="l00388" name="l00388"></a><span class="lineno">  388</span> </div>
<div class="line"><a id="l00389" name="l00389"></a><span class="lineno">  389</span>  <span class="keyword">const</span> <span class="keywordtype">size_t</span> limit = <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a43c42121d48a55a43a48e3a84bd10595">opt</a>(<a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#ade08f0457943f803c23b55649139c0d3">m_opts</a>.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Node__options.html#a88c732d10914f406cc43a2c20cc800c6">m_dyn_low_lvl_max_packet_size</a>);</div>
<div class="line"><a id="l00390" name="l00390"></a><span class="lineno">  390</span>  <span class="keywordflow">if</span> (bytes_to_send &gt; limit)</div>
<div class="line"><a id="l00391" name="l00391"></a><span class="lineno">  391</span>  {</div>
<div class="line"><a id="l00392" name="l00392"></a><span class="lineno">  392</span>    <span class="comment">// Bad and rare enough for a warning.</span></div>
<div class="line"><a id="l00393" name="l00393"></a><span class="lineno">  393</span>    <a class="code hl_define" href="log_8hpp.html#a626c7dc4d3b4dc0b32a8aac8624d66bc">FLOW_LOG_WARNING</a>(<span class="stringliteral">&quot;Tried to send low-level packet but before doing so detected &quot;</span></div>
<div class="line"><a id="l00394" name="l00394"></a><span class="lineno">  394</span>                     <span class="stringliteral">&quot;serialized size [&quot;</span> &lt;&lt; bytes_to_send &lt;&lt; <span class="stringliteral">&quot;] exceeds limit [&quot;</span> &lt;&lt; limit &lt;&lt; <span class="stringliteral">&quot;]; &quot;</span></div>
<div class="line"><a id="l00395" name="l00395"></a><span class="lineno">  395</span>                     <span class="stringliteral">&quot;check max-block-size and low-lvl-max-packet-size options!  Serialized packet: &quot;</span></div>
<div class="line"><a id="l00396" name="l00396"></a><span class="lineno">  396</span>                     <span class="stringliteral">&quot;[\n&quot;</span> &lt;&lt; packet-&gt;m_concise_ostream_manip &lt;&lt; <span class="stringliteral">&quot;].&quot;</span>);</div>
<div class="line"><a id="l00397" name="l00397"></a><span class="lineno">  397</span>    <span class="comment">// However, full packets should not be logged unless DATA log level allowed, period.</span></div>
<div class="line"><a id="l00398" name="l00398"></a><span class="lineno">  398</span>    <a class="code hl_define" href="log_8hpp.html#a07561c1c3f16b6c2c7335a09e25298c9">FLOW_LOG_DATA</a>(<span class="stringliteral">&quot;Detailed serialized packet details from preceding warning: &quot;</span></div>
<div class="line"><a id="l00399" name="l00399"></a><span class="lineno">  399</span>                  <span class="stringliteral">&quot;[\n&quot;</span> &lt;&lt; packet-&gt;m_verbose_ostream_manip &lt;&lt; <span class="stringliteral">&quot;].&quot;</span>);</div>
<div class="line"><a id="l00400" name="l00400"></a><span class="lineno">  400</span> </div>
<div class="line"><a id="l00401" name="l00401"></a><span class="lineno">  401</span>    <span class="comment">// Short-circuit this, since no send occurred.</span></div>
<div class="line"><a id="l00402" name="l00402"></a><span class="lineno">  402</span>    sock-&gt;m_snd_stats.low_lvl_packet_xfer_completed(packet_type_id, bytes_to_send, 0);</div>
<div class="line"><a id="l00403" name="l00403"></a><span class="lineno">  403</span>    <span class="keywordflow">return</span>;</div>
<div class="line"><a id="l00404" name="l00404"></a><span class="lineno">  404</span>  }</div>
<div class="line"><a id="l00405" name="l00405"></a><span class="lineno">  405</span>  <span class="comment">// else</span></div>
<div class="line"><a id="l00406" name="l00406"></a><span class="lineno">  406</span> </div>
<div class="line"><a id="l00407" name="l00407"></a><span class="lineno">  407</span>  <span class="comment">// Initiate asynchronous send.</span></div>
<div class="line"><a id="l00408" name="l00408"></a><span class="lineno">  408</span>  <a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#a9b9e0224c17190e99debdfd81640bf89">m_low_lvl_sock</a>.async_send_to(raw_bufs,</div>
<div class="line"><a id="l00409" name="l00409"></a><span class="lineno">  409</span>                               low_lvl_remote_endpoint,</div>
<div class="line"><a id="l00410" name="l00410"></a><span class="lineno">  410</span>                               [<span class="keyword">this</span>, sock, packet, bytes_to_send](<span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow.html#a29eaaa9d0fac4ce87d8b969222dbed09">Error_code</a>&amp; sys_err_code, <span class="keywordtype">size_t</span> n_sent)</div>
<div class="line"><a id="l00411" name="l00411"></a><span class="lineno">  411</span>  {</div>
<div class="line"><a id="l00412" name="l00412"></a><span class="lineno">  412</span>    <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a0055a25892990585561e69ceb02214d5">low_lvl_packet_sent</a>(sock, packet, bytes_to_send, sys_err_code, n_sent);</div>
<div class="line"><a id="l00413" name="l00413"></a><span class="lineno">  413</span>  });</div>
<div class="line"><a id="l00414" name="l00414"></a><span class="lineno">  414</span>} <span class="comment">// Node::async_low_lvl_packet_send_impl()</span></div>
<div class="line"><a id="l00415" name="l00415"></a><span class="lineno">  415</span> </div>
<div class="line"><a id="l00416" name="l00416"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#a2983d0d8a4d4eb5c3402f63f68ef44d8">  416</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a2983d0d8a4d4eb5c3402f63f68ef44d8">Node::mark_data_packet_sent</a>(<a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Peer_socket::Ptr</a> sock, <span class="keyword">const</span> <a class="code hl_class" href="classflow_1_1net__flow_1_1Sequence__number.html">Sequence_number</a>&amp; seq_num)</div>
<div class="line"><a id="l00417" name="l00417"></a><span class="lineno">  417</span>{</div>
<div class="line"><a id="l00418" name="l00418"></a><span class="lineno">  418</span>  <span class="keyword">using </span>boost::chrono::microseconds;</div>
<div class="line"><a id="l00419" name="l00419"></a><span class="lineno">  419</span>  <span class="keyword">using </span>boost::chrono::round;</div>
<div class="line"><a id="l00420" name="l00420"></a><span class="lineno">  420</span> </div>
<div class="line"><a id="l00421" name="l00421"></a><span class="lineno">  421</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00422" name="l00422"></a><span class="lineno">  422</span> </div>
<div class="line"><a id="l00423" name="l00423"></a><span class="lineno">  423</span>  <a class="code hl_typedef" href="classflow_1_1net__flow_1_1Peer__socket.html#ab7dc3b752b67295c82919c2f87aed60b">Peer_socket::Sent_pkt_ordered_by_when_iter</a> sent_pkt_it = sock-&gt;m_snd_flying_pkts_by_sent_when.find(seq_num);</div>
<div class="line"><a id="l00424" name="l00424"></a><span class="lineno">  424</span>  <span class="keywordflow">if</span> (sent_pkt_it == sock-&gt;m_snd_flying_pkts_by_sent_when.past_oldest())</div>
<div class="line"><a id="l00425" name="l00425"></a><span class="lineno">  425</span>  {</div>
<div class="line"><a id="l00426" name="l00426"></a><span class="lineno">  426</span>    <span class="comment">// We haven&#39;t even (or we have just) sent packet, and its In-Flight data already gone?  Very weird but not fatal.</span></div>
<div class="line"><a id="l00427" name="l00427"></a><span class="lineno">  427</span>    <a class="code hl_define" href="log_8hpp.html#a626c7dc4d3b4dc0b32a8aac8624d66bc">FLOW_LOG_WARNING</a>(<span class="stringliteral">&quot;Sending [DATA] packet over [&quot;</span> &lt;&lt; sock &lt;&lt; <span class="stringliteral">&quot;] with &quot;</span></div>
<div class="line"><a id="l00428" name="l00428"></a><span class="lineno">  428</span>                     <span class="stringliteral">&quot;sequence number [&quot;</span> &lt;&lt; seq_num &lt;&lt; <span class="stringliteral">&quot;] but cannot find corresponding Sent_packet.  &quot;</span></div>
<div class="line"><a id="l00429" name="l00429"></a><span class="lineno">  429</span>                     <span class="stringliteral">&quot;Cannot deal with some of the related data structures; still sending.  Bug?&quot;</span>);</div>
<div class="line"><a id="l00430" name="l00430"></a><span class="lineno">  430</span>    <span class="comment">// @todo Reading this now, it seems more than &quot;very weird.&quot; Possibly it should be an assert() or at least RST?</span></div>
<div class="line"><a id="l00431" name="l00431"></a><span class="lineno">  431</span>    <span class="keywordflow">return</span>;</div>
<div class="line"><a id="l00432" name="l00432"></a><span class="lineno">  432</span>  }</div>
<div class="line"><a id="l00433" name="l00433"></a><span class="lineno">  433</span>  <span class="comment">// else</span></div>
<div class="line"><a id="l00434" name="l00434"></a><span class="lineno">  434</span> </div>
<div class="line"><a id="l00435" name="l00435"></a><span class="lineno">  435</span>  <a class="code hl_struct" href="structflow_1_1net__flow_1_1Peer__socket_1_1Sent__packet.html">Peer_socket::Sent_packet</a>&amp; pkt = *sent_pkt_it-&gt;second;</div>
<div class="line"><a id="l00436" name="l00436"></a><span class="lineno">  436</span> </div>
<div class="line"><a id="l00437" name="l00437"></a><span class="lineno">  437</span>  <span class="comment">/* Mark the given packet as being sent right now.  The following structures depend on this</span></div>
<div class="line"><a id="l00438" name="l00438"></a><span class="lineno">  438</span><span class="comment">   * event:</span></div>
<div class="line"><a id="l00439" name="l00439"></a><span class="lineno">  439</span><span class="comment">   *</span></div>
<div class="line"><a id="l00440" name="l00440"></a><span class="lineno">  440</span><span class="comment">   *   - pkt.m_sent_when: Obviously needs to be marked with current time stamp (for RTT calculation</span></div>
<div class="line"><a id="l00441" name="l00441"></a><span class="lineno">  441</span><span class="comment">   *     later, among other things).</span></div>
<div class="line"><a id="l00442" name="l00442"></a><span class="lineno">  442</span><span class="comment">   *   - sock-&gt;m_snd_drop_timer: Since Drop_timer has an event for a packet becoming In-flight in the</span></div>
<div class="line"><a id="l00443" name="l00443"></a><span class="lineno">  443</span><span class="comment">   *     sense that it was actually sent over wire, which is exactly what&#39;s happening.</span></div>
<div class="line"><a id="l00444" name="l00444"></a><span class="lineno">  444</span><span class="comment">   *     So we call that.</span></div>
<div class="line"><a id="l00445" name="l00445"></a><span class="lineno">  445</span><span class="comment">   *   - Idle Timer: m_snd_last_data_sent_when is the m_sent_when value of the last DATA packet sent.</span></div>
<div class="line"><a id="l00446" name="l00446"></a><span class="lineno">  446</span><span class="comment">   *     Obviously we set that here. */</span></div>
<div class="line"><a id="l00447" name="l00447"></a><span class="lineno">  447</span> </div>
<div class="line"><a id="l00448" name="l00448"></a><span class="lineno">  448</span>  <span class="comment">// Get current time; and record it for Idle Timeout calculation.  See top of send_worker() for how it is used.</span></div>
<div class="line"><a id="l00449" name="l00449"></a><span class="lineno">  449</span>  <span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow.html#a9d9cc2eeb10d398cff5591d446b763b8">Fine_time_pt</a>&amp; now = sock-&gt;m_snd_last_data_sent_when = Fine_clock::now();</div>
<div class="line"><a id="l00450" name="l00450"></a><span class="lineno">  450</span>  <span class="keyword">const</span> <span class="keywordtype">size_t</span> cwnd = sock-&gt;m_snd_cong_ctl-&gt;congestion_window_bytes();</div>
<div class="line"><a id="l00451" name="l00451"></a><span class="lineno">  451</span>  <span class="keyword">auto</span>&amp; last_send_attempt = pkt.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Peer__socket_1_1Sent__packet.html#a3acccd2fcd0f4a0c5f7f0c2d6ad31c2e">m_sent_when</a>.back();</div>
<div class="line"><a id="l00452" name="l00452"></a><span class="lineno">  452</span>  <span class="keyword">const</span> <a class="code hl_typedef" href="classflow_1_1net__flow_1_1Peer__socket.html#a797a524b73e2626fcfefe1be64f4d481">Peer_socket::order_num_t</a> order_num = last_send_attempt.m_order_num;</div>
<div class="line"><a id="l00453" name="l00453"></a><span class="lineno">  453</span>  <span class="keyword">auto</span> <span class="keyword">const</span> logger_ptr = <a class="code hl_function" href="classflow_1_1log_1_1Log__context.html#af4db395042b065b00398cd59845dcb4d">get_logger</a>();</div>
<div class="line"><a id="l00454" name="l00454"></a><span class="lineno">  454</span>  <span class="keywordflow">if</span> (logger_ptr &amp;&amp; logger_ptr-&gt;should_log(<a class="code hl_enumvalue" href="namespaceflow_1_1log.html#a5c602384e6fe9d7642740ac2fae4ad04a3ee808b04c34bf6c110e3754de277f7b">log::Sev::S_TRACE</a>, <a class="code hl_function" href="classflow_1_1log_1_1Log__context.html#a8a5fef42dc2cedf3c0d1a6e83beb2948">get_log_component</a>()))</div>
<div class="line"><a id="l00455" name="l00455"></a><span class="lineno">  455</span>  {</div>
<div class="line"><a id="l00456" name="l00456"></a><span class="lineno">  456</span>    <span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow.html#a9d9cc2eeb10d398cff5591d446b763b8">Fine_time_pt</a> prev_sent_when = pkt.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Peer__socket_1_1Sent__packet.html#a3acccd2fcd0f4a0c5f7f0c2d6ad31c2e">m_sent_when</a>.back().m_sent_time;</div>
<div class="line"><a id="l00457" name="l00457"></a><span class="lineno">  457</span>    <span class="comment">// REPLACE the values.  Note m_order_num is just left alone.</span></div>
<div class="line"><a id="l00458" name="l00458"></a><span class="lineno">  458</span>    last_send_attempt.m_sent_time = now;</div>
<div class="line"><a id="l00459" name="l00459"></a><span class="lineno">  459</span>    last_send_attempt.m_sent_cwnd_bytes = cwnd;</div>
<div class="line"><a id="l00460" name="l00460"></a><span class="lineno">  460</span>    <span class="keyword">const</span> microseconds diff = round&lt;microseconds&gt;(now - prev_sent_when);</div>
<div class="line"><a id="l00461" name="l00461"></a><span class="lineno">  461</span> </div>
<div class="line"><a id="l00462" name="l00462"></a><span class="lineno">  462</span>    <a class="code hl_define" href="log_8hpp.html#a6b3b2cd7ddabf90a7c6ccd104a042cc6">FLOW_LOG_TRACE_WITHOUT_CHECKING</a></div>
<div class="line"><a id="l00463" name="l00463"></a><span class="lineno">  463</span>      (<span class="stringliteral">&quot;Sending/sent [DATA] packet over [&quot;</span> &lt;&lt; sock &lt;&lt; <span class="stringliteral">&quot;] with &quot;</span></div>
<div class="line"><a id="l00464" name="l00464"></a><span class="lineno">  464</span>       <span class="stringliteral">&quot;sequence number [&quot;</span> &lt;&lt; seq_num &lt;&lt; <span class="stringliteral">&quot;] order_num [&quot;</span> &lt;&lt; order_num &lt;&lt; <span class="stringliteral">&quot;].  Send timestamp changed from &quot;</span></div>
<div class="line"><a id="l00465" name="l00465"></a><span class="lineno">  465</span>       <span class="stringliteral">&quot;[&quot;</span> &lt;&lt; prev_sent_when &lt;&lt; <span class="stringliteral">&quot;] -&gt; [&quot;</span> &lt;&lt; now &lt;&lt; <span class="stringliteral">&quot;]; difference [&quot;</span> &lt;&lt; diff &lt;&lt; <span class="stringliteral">&quot;].&quot;</span>);</div>
<div class="line"><a id="l00466" name="l00466"></a><span class="lineno">  466</span>  }</div>
<div class="line"><a id="l00467" name="l00467"></a><span class="lineno">  467</span>  <span class="keywordflow">else</span></div>
<div class="line"><a id="l00468" name="l00468"></a><span class="lineno">  468</span>  {</div>
<div class="line"><a id="l00469" name="l00469"></a><span class="lineno">  469</span>    <span class="comment">// Same but no logging.</span></div>
<div class="line"><a id="l00470" name="l00470"></a><span class="lineno">  470</span>    last_send_attempt.m_sent_time = now;</div>
<div class="line"><a id="l00471" name="l00471"></a><span class="lineno">  471</span>    last_send_attempt.m_sent_cwnd_bytes = cwnd;</div>
<div class="line"><a id="l00472" name="l00472"></a><span class="lineno">  472</span>  }</div>
<div class="line"><a id="l00473" name="l00473"></a><span class="lineno">  473</span> </div>
<div class="line"><a id="l00474" name="l00474"></a><span class="lineno">  474</span>  <span class="comment">/* Inform this guy as required, now that packet has actually been sent off over the wire (not merely In-flight</span></div>
<div class="line"><a id="l00475" name="l00475"></a><span class="lineno">  475</span><span class="comment">   * by the Peer_socket definition, wherein it&#39;s been placed into sock-&gt;m_snd_flying_pkts*). */</span></div>
<div class="line"><a id="l00476" name="l00476"></a><span class="lineno">  476</span>  <span class="keyword">const</span> <a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Drop_timer::Ptr</a> drop_timer = sock-&gt;m_snd_drop_timer;</div>
<div class="line"><a id="l00477" name="l00477"></a><span class="lineno">  477</span>  drop_timer-&gt;start_contemporaneous_events();</div>
<div class="line"><a id="l00478" name="l00478"></a><span class="lineno">  478</span>  drop_timer-&gt;on_packet_in_flight(order_num);</div>
<div class="line"><a id="l00479" name="l00479"></a><span class="lineno">  479</span>  drop_timer-&gt;end_contemporaneous_events();</div>
<div class="line"><a id="l00480" name="l00480"></a><span class="lineno">  480</span>  <span class="comment">/* ^-- @todo Is it possible to smoosh a bunch of mark_data_packet_sent() calls into a single</span></div>
<div class="line"><a id="l00481" name="l00481"></a><span class="lineno">  481</span><span class="comment">   * start/end_contemporaneous_events() group? There are probably situations where 2 or more packets are sent</span></div>
<div class="line"><a id="l00482" name="l00482"></a><span class="lineno">  482</span><span class="comment">   * at a time which means ultimately mark_data_packet_sent() is called essentially contemporaneously.</span></div>
<div class="line"><a id="l00483" name="l00483"></a><span class="lineno">  483</span><span class="comment">   * However, at least as of this writing, mark_data_packet_sent() arises from a variety of situations, at least</span></div>
<div class="line"><a id="l00484" name="l00484"></a><span class="lineno">  484</span><span class="comment">   * some of which are &quot;singular&quot; and not really contemporaneous with any other packet(s) being sent.</span></div>
<div class="line"><a id="l00485" name="l00485"></a><span class="lineno">  485</span><span class="comment">   * So to isolate the cases where that is in fact true (2+ calls to us are contemporaneous) is certainly non-trivial,</span></div>
<div class="line"><a id="l00486" name="l00486"></a><span class="lineno">  486</span><span class="comment">   * and the code to do it would be... highly technical and hard to maintain. So I wouldn&#39;t trip over myself to</span></div>
<div class="line"><a id="l00487" name="l00487"></a><span class="lineno">  487</span><span class="comment">   * attempt this; and actually whatever book-keeping might be necessary to pull it off might itself have</span></div>
<div class="line"><a id="l00488" name="l00488"></a><span class="lineno">  488</span><span class="comment">   * some performance cost. */</span></div>
<div class="line"><a id="l00489" name="l00489"></a><span class="lineno">  489</span>} <span class="comment">// Node::mark_data_packet_sent()</span></div>
<div class="line"><a id="l00490" name="l00490"></a><span class="lineno">  490</span> </div>
<div class="line"><a id="l00491" name="l00491"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#a0055a25892990585561e69ceb02214d5">  491</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a0055a25892990585561e69ceb02214d5">Node::low_lvl_packet_sent</a>(<a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Peer_socket::Ptr</a> sock, <a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aef7998db71c60eeb5d1e3d1a97c14886">Low_lvl_packet::Const_ptr</a> packet,</div>
<div class="line"><a id="l00492" name="l00492"></a><span class="lineno">  492</span>                               <span class="keywordtype">size_t</span> bytes_expected_transferred, <span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow.html#a29eaaa9d0fac4ce87d8b969222dbed09">Error_code</a>&amp; sys_err_code,</div>
<div class="line"><a id="l00493" name="l00493"></a><span class="lineno">  493</span>                               <span class="keywordtype">size_t</span> bytes_transferred)</div>
<div class="line"><a id="l00494" name="l00494"></a><span class="lineno">  494</span>{</div>
<div class="line"><a id="l00495" name="l00495"></a><span class="lineno">  495</span>  <span class="keyword">using </span>std::numeric_limits;</div>
<div class="line"><a id="l00496" name="l00496"></a><span class="lineno">  496</span> </div>
<div class="line"><a id="l00497" name="l00497"></a><span class="lineno">  497</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00498" name="l00498"></a><span class="lineno">  498</span> </div>
<div class="line"><a id="l00499" name="l00499"></a><span class="lineno">  499</span>  <span class="keyword">const</span> <span class="keyword">auto</span>&amp; packet_ref = *packet;</div>
<div class="line"><a id="l00500" name="l00500"></a><span class="lineno">  500</span>  <span class="keyword">const</span> <span class="keyword">auto</span>&amp; packet_type_id = <span class="keyword">typeid</span>(packet_ref);</div>
<div class="line"><a id="l00501" name="l00501"></a><span class="lineno">  501</span> </div>
<div class="line"><a id="l00502" name="l00502"></a><span class="lineno">  502</span>  <span class="comment">// Note: we don&#39;t save `packet` anywhere, so the memory will finally be freed, when we&#39;re done here.</span></div>
<div class="line"><a id="l00503" name="l00503"></a><span class="lineno">  503</span> </div>
<div class="line"><a id="l00504" name="l00504"></a><span class="lineno">  504</span>  <span class="comment">// Log detailed info on packet but only if TRACE or DATA logging enabled.</span></div>
<div class="line"><a id="l00505" name="l00505"></a><span class="lineno">  505</span>  <span class="keyword">auto</span> <span class="keyword">const</span> logger_ptr = <a class="code hl_function" href="classflow_1_1log_1_1Log__context.html#af4db395042b065b00398cd59845dcb4d">get_logger</a>();</div>
<div class="line"><a id="l00506" name="l00506"></a><span class="lineno">  506</span>  <span class="keywordflow">if</span> (logger_ptr &amp;&amp; logger_ptr-&gt;should_log(<a class="code hl_enumvalue" href="namespaceflow_1_1log.html#a5c602384e6fe9d7642740ac2fae4ad04a3ee808b04c34bf6c110e3754de277f7b">log::Sev::S_TRACE</a>, <a class="code hl_function" href="classflow_1_1log_1_1Log__context.html#a8a5fef42dc2cedf3c0d1a6e83beb2948">get_log_component</a>()))</div>
<div class="line"><a id="l00507" name="l00507"></a><span class="lineno">  507</span>  {</div>
<div class="line"><a id="l00508" name="l00508"></a><span class="lineno">  508</span>    <span class="keywordflow">if</span> (logger_ptr-&gt;should_log(<a class="code hl_enumvalue" href="namespaceflow_1_1log.html#a5c602384e6fe9d7642740ac2fae4ad04a138fe9b2eb78ab00c84b21c4a6b48fed">log::Sev::S_DATA</a>, <a class="code hl_function" href="classflow_1_1log_1_1Log__context.html#a8a5fef42dc2cedf3c0d1a6e83beb2948">get_log_component</a>()))</div>
<div class="line"><a id="l00509" name="l00509"></a><span class="lineno">  509</span>    {</div>
<div class="line"><a id="l00510" name="l00510"></a><span class="lineno">  510</span>      <a class="code hl_define" href="log_8hpp.html#a9aa1d91ba9eb26345d4b83588ac489cb">FLOW_LOG_DATA_WITHOUT_CHECKING</a>(<span class="stringliteral">&quot;Tried to send low-level packet: &quot;</span></div>
<div class="line"><a id="l00511" name="l00511"></a><span class="lineno">  511</span>                                     <span class="stringliteral">&quot;[\n&quot;</span> &lt;&lt; packet-&gt;m_verbose_ostream_manip &lt;&lt; <span class="stringliteral">&quot;].&quot;</span>);</div>
<div class="line"><a id="l00512" name="l00512"></a><span class="lineno">  512</span>    }</div>
<div class="line"><a id="l00513" name="l00513"></a><span class="lineno">  513</span>    <span class="keywordflow">else</span></div>
<div class="line"><a id="l00514" name="l00514"></a><span class="lineno">  514</span>    {</div>
<div class="line"><a id="l00515" name="l00515"></a><span class="lineno">  515</span>      <a class="code hl_define" href="log_8hpp.html#a6b3b2cd7ddabf90a7c6ccd104a042cc6">FLOW_LOG_TRACE_WITHOUT_CHECKING</a>(<span class="stringliteral">&quot;Tried to send low-level packet: &quot;</span></div>
<div class="line"><a id="l00516" name="l00516"></a><span class="lineno">  516</span>                                      <span class="stringliteral">&quot;[\n&quot;</span> &lt;&lt; packet-&gt;m_concise_ostream_manip &lt;&lt; <span class="stringliteral">&quot;].&quot;</span>);</div>
<div class="line"><a id="l00517" name="l00517"></a><span class="lineno">  517</span>    }</div>
<div class="line"><a id="l00518" name="l00518"></a><span class="lineno">  518</span>  }</div>
<div class="line"><a id="l00519" name="l00519"></a><span class="lineno">  519</span> </div>
<div class="line"><a id="l00520" name="l00520"></a><span class="lineno">  520</span>  <span class="keywordflow">if</span> (sys_err_code)</div>
<div class="line"><a id="l00521" name="l00521"></a><span class="lineno">  521</span>  {</div>
<div class="line"><a id="l00522" name="l00522"></a><span class="lineno">  522</span>    <span class="comment">/* Weird it failed, since it&#39;s UDP.  Oh well.  (Update: One thing to watch out for is &quot;Message too long.&quot;!)</span></div>
<div class="line"><a id="l00523" name="l00523"></a><span class="lineno">  523</span><span class="comment">     * Note: One might be tempted to do something dire like close some sockets.  Several Flow connections might be</span></div>
<div class="line"><a id="l00524" name="l00524"></a><span class="lineno">  524</span><span class="comment">     * using the same UDP port, so should we close all of them or just the one to which this outgoing packet applies?</span></div>
<div class="line"><a id="l00525" name="l00525"></a><span class="lineno">  525</span><span class="comment">     * Even if we made some smart decision there, the failure to write could mean many things, so individual err_codes</span></div>
<div class="line"><a id="l00526" name="l00526"></a><span class="lineno">  526</span><span class="comment">     * would need to be examined with various actions taken depending on what happened.  (@todo Consider it.)</span></div>
<div class="line"><a id="l00527" name="l00527"></a><span class="lineno">  527</span><span class="comment">     * Without that, just ignore it as we would any lost packet.  Obviously, we deal with UDP&#39;s unreliability already.</span></div>
<div class="line"><a id="l00528" name="l00528"></a><span class="lineno">  528</span><span class="comment">     *</span></div>
<div class="line"><a id="l00529" name="l00529"></a><span class="lineno">  529</span><span class="comment">     * What about the specific error_codes &#39;would_block&#39; and &#39;try_again&#39; (in POSIX these probably both point to</span></div>
<div class="line"><a id="l00530" name="l00530"></a><span class="lineno">  530</span><span class="comment">     * code EAGAIN which == EWOULDBLOCK; in Windows possibly just the former, but to be safe both should be handled the</span></div>
<div class="line"><a id="l00531" name="l00531"></a><span class="lineno">  531</span><span class="comment">     * same way)?   Currently I am ignoring this possibility.  Why?  Answer: we specifically use async_send() to wait</span></div>
<div class="line"><a id="l00532" name="l00532"></a><span class="lineno">  532</span><span class="comment">     * for the socket to be ready for the trasmit operation before actually transmitting.  This _should_ ensure</span></div>
<div class="line"><a id="l00533" name="l00533"></a><span class="lineno">  533</span><span class="comment">     * that those two errors are _specifically_ avoided.  However, that&#39;s probably not a 100% safe assumption.</span></div>
<div class="line"><a id="l00534" name="l00534"></a><span class="lineno">  534</span><span class="comment">     * Even if the OS reports &quot;Writable&quot; at time T, a few microseconds later some resource might get used up</span></div>
<div class="line"><a id="l00535" name="l00535"></a><span class="lineno">  535</span><span class="comment">     * (unrelated to our activities); and by the time the actual send executes, we might get would_block/try_again.</span></div>
<div class="line"><a id="l00536" name="l00536"></a><span class="lineno">  536</span><span class="comment">     * Now, it&#39;s possible that since we do NOT use async_wait() instead of async_send() call -- meaning we let</span></div>
<div class="line"><a id="l00537" name="l00537"></a><span class="lineno">  537</span><span class="comment">     * boost.asio both wait for Writable *and* itself execute the write -- that it would hide any such</span></div>
<div class="line"><a id="l00538" name="l00538"></a><span class="lineno">  538</span><span class="comment">     * corner-case EAGAIN/etc. from us and just not call this handler in that case and retry later by itself.</span></div>
<div class="line"><a id="l00539" name="l00539"></a><span class="lineno">  539</span><span class="comment">     * However, I don&#39;t know if it does that.  @todo Therefore it would be safer to treat those error codes,</span></div>
<div class="line"><a id="l00540" name="l00540"></a><span class="lineno">  540</span><span class="comment">     * would_block and try_again, specially here.  Simply, we should retry the async_send(), meaning it will wait for</span></div>
<div class="line"><a id="l00541" name="l00541"></a><span class="lineno">  541</span><span class="comment">     * Writable again and execute it again later on.  One might worry that this might open the possibility of</span></div>
<div class="line"><a id="l00542" name="l00542"></a><span class="lineno">  542</span><span class="comment">     * more than one outstanding async_send().  If this happens, it&#39;s fine.  Even if boost.asio would be so rash as</span></div>
<div class="line"><a id="l00543" name="l00543"></a><span class="lineno">  543</span><span class="comment">     * to allow outstanding async_send()s to execute their handlers in a different order from the calls, it&#39;s not</span></div>
<div class="line"><a id="l00544" name="l00544"></a><span class="lineno">  544</span><span class="comment">     * a problem, as out-of-order datagrams are handled just fine by the other side.</span></div>
<div class="line"><a id="l00545" name="l00545"></a><span class="lineno">  545</span><span class="comment">     *</span></div>
<div class="line"><a id="l00546" name="l00546"></a><span class="lineno">  546</span><span class="comment">     * Note: I&#39;ve placed a more visible @todo in the class doc header.  So delete that if this gets done. */</span></div>
<div class="line"><a id="l00547" name="l00547"></a><span class="lineno">  547</span>    <a class="code hl_define" href="error_2error_8hpp.html#a019ee6e90802e444572dd78d312d6e23">FLOW_ERROR_SYS_ERROR_LOG_WARNING</a>();</div>
<div class="line"><a id="l00548" name="l00548"></a><span class="lineno">  548</span> </div>
<div class="line"><a id="l00549" name="l00549"></a><span class="lineno">  549</span>    <span class="keywordflow">if</span> (sock)</div>
<div class="line"><a id="l00550" name="l00550"></a><span class="lineno">  550</span>    {</div>
<div class="line"><a id="l00551" name="l00551"></a><span class="lineno">  551</span>      sock-&gt;m_snd_stats.low_lvl_packet_xfer_completed(packet_type_id);</div>
<div class="line"><a id="l00552" name="l00552"></a><span class="lineno">  552</span>    }</div>
<div class="line"><a id="l00553" name="l00553"></a><span class="lineno">  553</span> </div>
<div class="line"><a id="l00554" name="l00554"></a><span class="lineno">  554</span>    <span class="keywordflow">return</span>;</div>
<div class="line"><a id="l00555" name="l00555"></a><span class="lineno">  555</span>  }</div>
<div class="line"><a id="l00556" name="l00556"></a><span class="lineno">  556</span>  <span class="comment">// else</span></div>
<div class="line"><a id="l00557" name="l00557"></a><span class="lineno">  557</span> </div>
<div class="line"><a id="l00558" name="l00558"></a><span class="lineno">  558</span>  <span class="keywordflow">if</span> (sock)</div>
<div class="line"><a id="l00559" name="l00559"></a><span class="lineno">  559</span>  {</div>
<div class="line"><a id="l00560" name="l00560"></a><span class="lineno">  560</span>    sock-&gt;m_snd_stats.low_lvl_packet_xfer_completed(packet_type_id, bytes_expected_transferred, bytes_transferred);</div>
<div class="line"><a id="l00561" name="l00561"></a><span class="lineno">  561</span>  }</div>
<div class="line"><a id="l00562" name="l00562"></a><span class="lineno">  562</span> </div>
<div class="line"><a id="l00563" name="l00563"></a><span class="lineno">  563</span>  <span class="keywordflow">if</span> (bytes_transferred != bytes_expected_transferred)</div>
<div class="line"><a id="l00564" name="l00564"></a><span class="lineno">  564</span>  {</div>
<div class="line"><a id="l00565" name="l00565"></a><span class="lineno">  565</span>    <span class="comment">/* If a UDP send was partial, the datagram must have been too big, which we shouldn&#39;t have allowed; worth warning.</span></div>
<div class="line"><a id="l00566" name="l00566"></a><span class="lineno">  566</span><span class="comment">     * Update: Actually, at least on Mac with Boost 1.63 I&#39;ve seen these outcomes:</span></div>
<div class="line"><a id="l00567" name="l00567"></a><span class="lineno">  567</span><span class="comment">     *   - If message is too big byte-wise, it results in truthy sys_err_code.</span></div>
<div class="line"><a id="l00568" name="l00568"></a><span class="lineno">  568</span><span class="comment">     *   - If too many sub-buffers in scatter/gather buffer container passed to async_send_to() were passed,</span></div>
<div class="line"><a id="l00569" name="l00569"></a><span class="lineno">  569</span><span class="comment">     *     they may be silently truncated (!sys_err_code but the present condition is triggered).  Limit was observed to</span></div>
<div class="line"><a id="l00570" name="l00570"></a><span class="lineno">  570</span><span class="comment">     *     be ~64. */</span></div>
<div class="line"><a id="l00571" name="l00571"></a><span class="lineno">  571</span>    <a class="code hl_define" href="log_8hpp.html#a626c7dc4d3b4dc0b32a8aac8624d66bc">FLOW_LOG_WARNING</a>(<span class="stringliteral">&quot;Low-level packet sent, but only [&quot;</span> &lt;&lt; bytes_transferred &lt;&lt; <span class="stringliteral">&quot;] of &quot;</span></div>
<div class="line"><a id="l00572" name="l00572"></a><span class="lineno">  572</span>                     <span class="stringliteral">&quot;[&quot;</span> &lt;&lt; bytes_expected_transferred &lt;&lt; <span class="stringliteral">&quot;] bytes &quot;</span></div>
<div class="line"><a id="l00573" name="l00573"></a><span class="lineno">  573</span>                     <span class="stringliteral">&quot;were sent.  Internal error with packet size calculations?  More likely, did stack truncate?&quot;</span>);</div>
<div class="line"><a id="l00574" name="l00574"></a><span class="lineno">  574</span>    <span class="keywordflow">return</span>;</div>
<div class="line"><a id="l00575" name="l00575"></a><span class="lineno">  575</span>  }</div>
<div class="line"><a id="l00576" name="l00576"></a><span class="lineno">  576</span>  <span class="comment">// else</span></div>
<div class="line"><a id="l00577" name="l00577"></a><span class="lineno">  577</span>  <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Success.&quot;</span>);</div>
<div class="line"><a id="l00578" name="l00578"></a><span class="lineno">  578</span>} <span class="comment">// Node::low_lvl_packet_sent()</span></div>
<div class="line"><a id="l00579" name="l00579"></a><span class="lineno">  579</span> </div>
<div class="line"><a id="l00580" name="l00580"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#a74a6a82a1390a76e985593969fb47184">  580</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a74a6a82a1390a76e985593969fb47184">Node::async_no_sock_low_lvl_rst_send</a>(<a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aef7998db71c60eeb5d1e3d1a97c14886">Low_lvl_packet::Const_ptr</a> causing_packet,</div>
<div class="line"><a id="l00581" name="l00581"></a><span class="lineno">  581</span>                                          <span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow_1_1util.html#af1b28edb7bad743c1c77f6b982cc3196">util::Udp_endpoint</a>&amp; low_lvl_remote_endpoint)</div>
<div class="line"><a id="l00582" name="l00582"></a><span class="lineno">  582</span>{</div>
<div class="line"><a id="l00583" name="l00583"></a><span class="lineno">  583</span>  <span class="keyword">using </span>boost::asio::buffer;</div>
<div class="line"><a id="l00584" name="l00584"></a><span class="lineno">  584</span>  <span class="keyword">using </span>boost::shared_ptr;</div>
<div class="line"><a id="l00585" name="l00585"></a><span class="lineno">  585</span> </div>
<div class="line"><a id="l00586" name="l00586"></a><span class="lineno">  586</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00587" name="l00587"></a><span class="lineno">  587</span> </div>
<div class="line"><a id="l00588" name="l00588"></a><span class="lineno">  588</span>  <span class="comment">// Fill out typical info.</span></div>
<div class="line"><a id="l00589" name="l00589"></a><span class="lineno">  589</span>  <span class="keyword">auto</span> rst_base = Low_lvl_packet::create_uninit_packet_base&lt;Rst_packet&gt;(<a class="code hl_function" href="classflow_1_1log_1_1Log__context.html#af4db395042b065b00398cd59845dcb4d">get_logger</a>());</div>
<div class="line"><a id="l00590" name="l00590"></a><span class="lineno">  590</span>  <span class="comment">// No sequence number.  See m_seq_num for discussion.</span></div>
<div class="line"><a id="l00591" name="l00591"></a><span class="lineno">  591</span>  rst_base-&gt;m_packed.m_src_port = causing_packet-&gt;m_packed.m_dst_port; <span class="comment">// Right back atcha.</span></div>
<div class="line"><a id="l00592" name="l00592"></a><span class="lineno">  592</span>  rst_base-&gt;m_packed.m_dst_port = causing_packet-&gt;m_packed.m_src_port;</div>
<div class="line"><a id="l00593" name="l00593"></a><span class="lineno">  593</span>  rst_base-&gt;m_opt_rexmit_on = <span class="keyword">false</span>; <span class="comment">// Not used in RST packets, so set it to something.</span></div>
<div class="line"><a id="l00594" name="l00594"></a><span class="lineno">  594</span> </div>
<div class="line"><a id="l00595" name="l00595"></a><span class="lineno">  595</span>  <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a08a5752bf3848f31abbe753be47c1fcf">async_no_sock_low_lvl_packet_send</a>(low_lvl_remote_endpoint, rst_base);</div>
<div class="line"><a id="l00596" name="l00596"></a><span class="lineno">  596</span>  <span class="comment">// If that returned false: It&#39;s an RST, so there&#39;s no one to inform of an error anymore.  Oh well.</span></div>
<div class="line"><a id="l00597" name="l00597"></a><span class="lineno">  597</span>} <span class="comment">// Node::async_no_sock_low_lvl_rst_send()</span></div>
<div class="line"><a id="l00598" name="l00598"></a><span class="lineno">  598</span> </div>
<div class="line"><a id="l00599" name="l00599"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#af1c4623729d0a061200a386f7a651926">  599</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#af1c4623729d0a061200a386f7a651926">Node::async_sock_low_lvl_packet_send_paced</a>(<span class="keyword">const</span> <a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Peer_socket::Ptr</a>&amp; sock,</div>
<div class="line"><a id="l00600" name="l00600"></a><span class="lineno">  600</span>                                                <a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Low_lvl_packet::Ptr</a>&amp;&amp; packet)</div>
<div class="line"><a id="l00601" name="l00601"></a><span class="lineno">  601</span>{</div>
<div class="line"><a id="l00602" name="l00602"></a><span class="lineno">  602</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00603" name="l00603"></a><span class="lineno">  603</span> </div>
<div class="line"><a id="l00604" name="l00604"></a><span class="lineno">  604</span>  <span class="comment">// This is the general-purpose method for sending a packet along a well-defined connection (sock).</span></div>
<div class="line"><a id="l00605" name="l00605"></a><span class="lineno">  605</span> </div>
<div class="line"><a id="l00606" name="l00606"></a><span class="lineno">  606</span>  <span class="keyword">const</span> <span class="keyword">auto</span>&amp; packet_ref = *packet;</div>
<div class="line"><a id="l00607" name="l00607"></a><span class="lineno">  607</span>  <span class="keyword">const</span> <span class="keyword">auto</span>&amp; packet_type_id = <span class="keyword">typeid</span>(packet_ref);</div>
<div class="line"><a id="l00608" name="l00608"></a><span class="lineno">  608</span> </div>
<div class="line"><a id="l00609" name="l00609"></a><span class="lineno">  609</span>  sock-&gt;m_snd_stats.low_lvl_packet_xfer_requested(packet_type_id);</div>
<div class="line"><a id="l00610" name="l00610"></a><span class="lineno">  610</span> </div>
<div class="line"><a id="l00611" name="l00611"></a><span class="lineno">  611</span>  <span class="comment">// Fill out typical info.</span></div>
<div class="line"><a id="l00612" name="l00612"></a><span class="lineno">  612</span>  packet-&gt;m_packed.m_src_port = sock-&gt;m_local_port;</div>
<div class="line"><a id="l00613" name="l00613"></a><span class="lineno">  613</span>  packet-&gt;m_packed.m_dst_port = sock-&gt;remote_endpoint().m_flow_port;</div>
<div class="line"><a id="l00614" name="l00614"></a><span class="lineno">  614</span>  packet-&gt;m_opt_rexmit_on = sock-&gt;rexmit_on();</div>
<div class="line"><a id="l00615" name="l00615"></a><span class="lineno">  615</span> </div>
<div class="line"><a id="l00616" name="l00616"></a><span class="lineno">  616</span>  <span class="comment">/* Apply packet pacing, which tries to spread out bursts of packets to prevent loss.  For much</span></div>
<div class="line"><a id="l00617" name="l00617"></a><span class="lineno">  617</span><span class="comment">   * more detail, see struct Send_pacing_data comment. */</span></div>
<div class="line"><a id="l00618" name="l00618"></a><span class="lineno">  618</span> </div>
<div class="line"><a id="l00619" name="l00619"></a><span class="lineno">  619</span>  <span class="keywordflow">if</span> ((!sock-&gt;opt(sock-&gt;m_opts.m_st_snd_pacing_enabled)) ||</div>
<div class="line"><a id="l00620" name="l00620"></a><span class="lineno">  620</span>      (sock-&gt;m_snd_smoothed_round_trip_time == Fine_duration::zero()) ||</div>
<div class="line"><a id="l00621" name="l00621"></a><span class="lineno">  621</span>      (packet_type_id == <span class="keyword">typeid</span>(<a class="code hl_struct" href="structflow_1_1net__flow_1_1Rst__packet.html">Rst_packet</a>)))</div>
<div class="line"><a id="l00622" name="l00622"></a><span class="lineno">  622</span>  {</div>
<div class="line"><a id="l00623" name="l00623"></a><span class="lineno">  623</span>    <span class="comment">/* Per struct Send_pacing_data doc header, the pacing algorithm only begins once we have our</span></div>
<div class="line"><a id="l00624" name="l00624"></a><span class="lineno">  624</span><span class="comment">     * first round trip time (and thus SRTT); until then we send all packets as soon as possible.</span></div>
<div class="line"><a id="l00625" name="l00625"></a><span class="lineno">  625</span><span class="comment">     * Also pacing can just be disabled; in which case send all packets ASAP.</span></div>
<div class="line"><a id="l00626" name="l00626"></a><span class="lineno">  626</span><span class="comment">     *</span></div>
<div class="line"><a id="l00627" name="l00627"></a><span class="lineno">  627</span><span class="comment">     * Finally, if it&#39;s an RST, just send it ASAP.  This is discussed in more detail in the</span></div>
<div class="line"><a id="l00628" name="l00628"></a><span class="lineno">  628</span><span class="comment">     * Send_pacing_data struct header, but basically:  RST means packet is being CLOSED right now.</span></div>
<div class="line"><a id="l00629" name="l00629"></a><span class="lineno">  629</span><span class="comment">     * Queueing RST on the pacing queue means it may be sent after underlying socket is CLOSED.</span></div>
<div class="line"><a id="l00630" name="l00630"></a><span class="lineno">  630</span><span class="comment">     * Therefore we have to keep queue and all that machinery operating past socket being CLOSED.</span></div>
<div class="line"><a id="l00631" name="l00631"></a><span class="lineno">  631</span><span class="comment">     * As a rule of thumb, CLOSED =&gt; dead socket.  So just send RST right away.  This means it may</span></div>
<div class="line"><a id="l00632" name="l00632"></a><span class="lineno">  632</span><span class="comment">     * jump the queue ahead of DATA/ACK packets already there, but since it is an error condition</span></div>
<div class="line"><a id="l00633" name="l00633"></a><span class="lineno">  633</span><span class="comment">     * causing RST, we consider that OK (those packets will not be sent). */</span></div>
<div class="line"><a id="l00634" name="l00634"></a><span class="lineno">  634</span>    <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a0db2bb587e0407077b11368df075f0ff">async_sock_low_lvl_packet_send</a>(sock, std::move(packet), <span class="keyword">false</span>); <span class="comment">// false =&gt; not queued in pacing module.</span></div>
<div class="line"><a id="l00635" name="l00635"></a><span class="lineno">  635</span>    <span class="keywordflow">return</span>;</div>
<div class="line"><a id="l00636" name="l00636"></a><span class="lineno">  636</span>  }</div>
<div class="line"><a id="l00637" name="l00637"></a><span class="lineno">  637</span>  <span class="comment">// else pacing algorithm enabled and both can and must be used.</span></div>
<div class="line"><a id="l00638" name="l00638"></a><span class="lineno">  638</span> </div>
<div class="line"><a id="l00639" name="l00639"></a><span class="lineno">  639</span>  <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a0556af278dc07a43f109fbd74ac64d5e">sock_pacing_new_packet_ready</a>(sock, std::move(packet));</div>
<div class="line"><a id="l00640" name="l00640"></a><span class="lineno">  640</span>} <span class="comment">// Node::async_sock_low_lvl_packet_send_paced()</span></div>
<div class="line"><a id="l00641" name="l00641"></a><span class="lineno">  641</span> </div>
<div class="line"><a id="l00642" name="l00642"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#a0556af278dc07a43f109fbd74ac64d5e">  642</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a0556af278dc07a43f109fbd74ac64d5e">Node::sock_pacing_new_packet_ready</a>(<a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Peer_socket::Ptr</a> sock, <a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Low_lvl_packet::Ptr</a> packet)</div>
<div class="line"><a id="l00643" name="l00643"></a><span class="lineno">  643</span>{</div>
<div class="line"><a id="l00644" name="l00644"></a><span class="lineno">  644</span>  <span class="keyword">using </span>boost::chrono::duration_cast;</div>
<div class="line"><a id="l00645" name="l00645"></a><span class="lineno">  645</span>  <span class="keyword">using </span>boost::chrono::microseconds;</div>
<div class="line"><a id="l00646" name="l00646"></a><span class="lineno">  646</span>  <span class="keyword">using </span>boost::static_pointer_cast;</div>
<div class="line"><a id="l00647" name="l00647"></a><span class="lineno">  647</span>  <span class="keyword">using </span>boost::dynamic_pointer_cast;</div>
<div class="line"><a id="l00648" name="l00648"></a><span class="lineno">  648</span>  <span class="keyword">using </span>boost::shared_ptr;</div>
<div class="line"><a id="l00649" name="l00649"></a><span class="lineno">  649</span> </div>
<div class="line"><a id="l00650" name="l00650"></a><span class="lineno">  650</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00651" name="l00651"></a><span class="lineno">  651</span> </div>
<div class="line"><a id="l00652" name="l00652"></a><span class="lineno">  652</span>  <span class="keyword">const</span> <span class="keyword">auto</span>&amp; packet_ref = *packet;</div>
<div class="line"><a id="l00653" name="l00653"></a><span class="lineno">  653</span>  <span class="keyword">const</span> <span class="keyword">auto</span>&amp; packet_type_id = <span class="keyword">typeid</span>(packet_ref);</div>
<div class="line"><a id="l00654" name="l00654"></a><span class="lineno">  654</span> </div>
<div class="line"><a id="l00655" name="l00655"></a><span class="lineno">  655</span>  <span class="comment">// For brevity and a bit of speed.</span></div>
<div class="line"><a id="l00656" name="l00656"></a><span class="lineno">  656</span>  <a class="code hl_struct" href="structflow_1_1net__flow_1_1Send__pacing__data.html">Send_pacing_data</a>&amp; pacing = sock-&gt;m_snd_pacing_data;</div>
<div class="line"><a id="l00657" name="l00657"></a><span class="lineno">  657</span> </div>
<div class="line"><a id="l00658" name="l00658"></a><span class="lineno">  658</span>  <span class="keyword">const</span> <span class="keywordtype">bool</span> is_data_packet = packet_type_id == <span class="keyword">typeid</span>(<a class="code hl_struct" href="structflow_1_1net__flow_1_1Data__packet.html">Data_packet</a>);</div>
<div class="line"><a id="l00659" name="l00659"></a><span class="lineno">  659</span>  <span class="comment">// For logging, get the first sequence number mentioned (depending on whether it&#39;s DATA or ACK).</span></div>
<div class="line"><a id="l00660" name="l00660"></a><span class="lineno">  660</span>  <a class="code hl_class" href="classflow_1_1net__flow_1_1Sequence__number.html">Sequence_number</a> init_seq_num;</div>
<div class="line"><a id="l00661" name="l00661"></a><span class="lineno">  661</span>  shared_ptr&lt;const Data_packet&gt; data;</div>
<div class="line"><a id="l00662" name="l00662"></a><span class="lineno">  662</span>  <span class="keywordflow">if</span> (is_data_packet)</div>
<div class="line"><a id="l00663" name="l00663"></a><span class="lineno">  663</span>  {</div>
<div class="line"><a id="l00664" name="l00664"></a><span class="lineno">  664</span>    init_seq_num = static_pointer_cast&lt;const Data_packet&gt;(packet)-&gt;m_seq_num;</div>
<div class="line"><a id="l00665" name="l00665"></a><span class="lineno">  665</span>  }</div>
<div class="line"><a id="l00666" name="l00666"></a><span class="lineno">  666</span>  <span class="keywordflow">else</span></div>
<div class="line"><a id="l00667" name="l00667"></a><span class="lineno">  667</span>  {</div>
<div class="line"><a id="l00668" name="l00668"></a><span class="lineno">  668</span>    <span class="keyword">const</span> <span class="keyword">auto</span>&amp; acked_packets = static_pointer_cast&lt;const Ack_packet&gt;(packet)-&gt;m_rcv_acked_packets;</div>
<div class="line"><a id="l00669" name="l00669"></a><span class="lineno">  669</span>    <span class="keywordflow">if</span> (!acked_packets.empty())</div>
<div class="line"><a id="l00670" name="l00670"></a><span class="lineno">  670</span>    {</div>
<div class="line"><a id="l00671" name="l00671"></a><span class="lineno">  671</span>      init_seq_num = acked_packets.front()-&gt;m_seq_num;</div>
<div class="line"><a id="l00672" name="l00672"></a><span class="lineno">  672</span>    }</div>
<div class="line"><a id="l00673" name="l00673"></a><span class="lineno">  673</span>  }</div>
<div class="line"><a id="l00674" name="l00674"></a><span class="lineno">  674</span> </div>
<div class="line"><a id="l00675" name="l00675"></a><span class="lineno">  675</span>  <span class="keyword">const</span> <span class="keywordtype">bool</span> q_was_empty = pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.empty();</div>
<div class="line"><a id="l00676" name="l00676"></a><span class="lineno">  676</span> </div>
<div class="line"><a id="l00677" name="l00677"></a><span class="lineno">  677</span>  <span class="comment">/* No matter what, we can&#39;t send packet before the ones already in the queue, so push onto end</span></div>
<div class="line"><a id="l00678" name="l00678"></a><span class="lineno">  678</span><span class="comment">   * of the queue.  (If queue is currently empty, then that is not a special case; we may well</span></div>
<div class="line"><a id="l00679" name="l00679"></a><span class="lineno">  679</span><span class="comment">   * immediately pop and send it below.) */</span></div>
<div class="line"><a id="l00680" name="l00680"></a><span class="lineno">  680</span>  pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.push_back(packet);</div>
<div class="line"><a id="l00681" name="l00681"></a><span class="lineno">  681</span> </div>
<div class="line"><a id="l00682" name="l00682"></a><span class="lineno">  682</span>  <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Pacing: On [&quot;</span> &lt;&lt; sock &lt;&lt; <span class="stringliteral">&quot;] packet of type [&quot;</span> &lt;&lt; packet-&gt;m_type_ostream_manip &lt;&lt; <span class="stringliteral">&quot;] &quot;</span></div>
<div class="line"><a id="l00683" name="l00683"></a><span class="lineno">  683</span>                 <span class="stringliteral">&quot;is newly available for sending; pushed onto queue; queue size [&quot;</span> &lt;&lt; pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.size() &lt;&lt; <span class="stringliteral">&quot;]; &quot;</span></div>
<div class="line"><a id="l00684" name="l00684"></a><span class="lineno">  684</span>                 <span class="stringliteral">&quot;initial sequence number [&quot;</span> &lt;&lt; init_seq_num &lt;&lt; <span class="stringliteral">&quot;].&quot;</span>);</div>
<div class="line"><a id="l00685" name="l00685"></a><span class="lineno">  685</span> </div>
<div class="line"><a id="l00686" name="l00686"></a><span class="lineno">  686</span>  <span class="keywordflow">if</span> (!q_was_empty)</div>
<div class="line"><a id="l00687" name="l00687"></a><span class="lineno">  687</span>  {</div>
<div class="line"><a id="l00688" name="l00688"></a><span class="lineno">  688</span>    <span class="keyword">const</span> <span class="keyword">auto</span> data = dynamic_pointer_cast&lt;const Data_packet&gt;(pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.front());</div>
<div class="line"><a id="l00689" name="l00689"></a><span class="lineno">  689</span>    <span class="comment">// !data if it&#39;s not a DATA packet.</span></div>
<div class="line"><a id="l00690" name="l00690"></a><span class="lineno">  690</span> </div>
<div class="line"><a id="l00691" name="l00691"></a><span class="lineno">  691</span>    assert((!data) || (pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#a9cfd632911c3c4acd8e048dae28ded46">m_bytes_allowed_this_slice</a> &lt; data-&gt;m_data.size()));</div>
<div class="line"><a id="l00692" name="l00692"></a><span class="lineno">  692</span> </div>
<div class="line"><a id="l00693" name="l00693"></a><span class="lineno">  693</span>    <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Pacing: On [&quot;</span> &lt;&lt; sock &lt;&lt; <span class="stringliteral">&quot;]: was already in progress; queued and done.&quot;</span>);</div>
<div class="line"><a id="l00694" name="l00694"></a><span class="lineno">  694</span> </div>
<div class="line"><a id="l00695" name="l00695"></a><span class="lineno">  695</span>    <span class="comment">/* There were already packets in the queue, so the timer is running; certainly we can&#39;t send</span></div>
<div class="line"><a id="l00696" name="l00696"></a><span class="lineno">  696</span><span class="comment">     * packet ahead of those in the queue, so push on the back of the queue above.  Should we send</span></div>
<div class="line"><a id="l00697" name="l00697"></a><span class="lineno">  697</span><span class="comment">     * the head packet now?  No; if the last sock_pacing_new_packet_ready() or</span></div>
<div class="line"><a id="l00698" name="l00698"></a><span class="lineno">  698</span><span class="comment">     * sock_pacing_time_slice_end() left a non-empty queue, then the timer has been set to fire when</span></div>
<div class="line"><a id="l00699" name="l00699"></a><span class="lineno">  699</span><span class="comment">     * the slice ends, and more packets can be sent.  Done. */</span></div>
<div class="line"><a id="l00700" name="l00700"></a><span class="lineno">  700</span>    <span class="keywordflow">return</span>;</div>
<div class="line"><a id="l00701" name="l00701"></a><span class="lineno">  701</span>  }</div>
<div class="line"><a id="l00702" name="l00702"></a><span class="lineno">  702</span>  <span class="comment">// else if (q_was_empty)</span></div>
<div class="line"><a id="l00703" name="l00703"></a><span class="lineno">  703</span> </div>
<div class="line"><a id="l00704" name="l00704"></a><span class="lineno">  704</span>  <span class="comment">// We have just pushed the sole packet on the queue.</span></div>
<div class="line"><a id="l00705" name="l00705"></a><span class="lineno">  705</span> </div>
<div class="line"><a id="l00706" name="l00706"></a><span class="lineno">  706</span>  <span class="keywordflow">if</span> (!is_data_packet)</div>
<div class="line"><a id="l00707" name="l00707"></a><span class="lineno">  707</span>  {</div>
<div class="line"><a id="l00708" name="l00708"></a><span class="lineno">  708</span>    assert(packet_type_id == <span class="keyword">typeid</span>(<a class="code hl_struct" href="structflow_1_1net__flow_1_1Ack__packet.html">Ack_packet</a>)); <span class="comment">// Sanity check.</span></div>
<div class="line"><a id="l00709" name="l00709"></a><span class="lineno">  709</span>    <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Pacing: On [&quot;</span> &lt;&lt; sock &lt;&lt; <span class="stringliteral">&quot;]: due to packet type, sending immediately since at head of queue; &quot;</span></div>
<div class="line"><a id="l00710" name="l00710"></a><span class="lineno">  710</span>                   <span class="stringliteral">&quot;queue empty again.&quot;</span>);</div>
<div class="line"><a id="l00711" name="l00711"></a><span class="lineno">  711</span> </div>
<div class="line"><a id="l00712" name="l00712"></a><span class="lineno">  712</span>    pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.clear();</div>
<div class="line"><a id="l00713" name="l00713"></a><span class="lineno">  713</span> </div>
<div class="line"><a id="l00714" name="l00714"></a><span class="lineno">  714</span>    <span class="comment">/* Per discussion in struct Send_pacing_data doc header, if it&#39;s a non-DATA packet, then --</span></div>
<div class="line"><a id="l00715" name="l00715"></a><span class="lineno">  715</span><span class="comment">     * other than not being allowed to be re-ordered -- it does not &quot;count&quot; as a paced packet.  That</span></div>
<div class="line"><a id="l00716" name="l00716"></a><span class="lineno">  716</span><span class="comment">     * is, it is &quot;worth&quot; zero bytes when compared to m_bytes_allowed_this_slice and should be sent</span></div>
<div class="line"><a id="l00717" name="l00717"></a><span class="lineno">  717</span><span class="comment">     * as soon as it is at the head of the queue (which it is here, since we just placed it as the</span></div>
<div class="line"><a id="l00718" name="l00718"></a><span class="lineno">  718</span><span class="comment">     * sole element on the queue).  Since it doesn&#39;t count against m_bytes_allowed_this_slice, the</span></div>
<div class="line"><a id="l00719" name="l00719"></a><span class="lineno">  719</span><span class="comment">     * pacing timing is irrelevant to it, and based on the &quot;send ASAP&quot; rule, we send it now. */</span></div>
<div class="line"><a id="l00720" name="l00720"></a><span class="lineno">  720</span>    <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a0db2bb587e0407077b11368df075f0ff">async_sock_low_lvl_packet_send</a>(sock, std::move(packet), <span class="keyword">false</span>); <span class="comment">// false =&gt; not queued in pacing module.</span></div>
<div class="line"><a id="l00721" name="l00721"></a><span class="lineno">  721</span>    <span class="keywordflow">return</span>;</div>
<div class="line"><a id="l00722" name="l00722"></a><span class="lineno">  722</span>  }</div>
<div class="line"><a id="l00723" name="l00723"></a><span class="lineno">  723</span>  <span class="comment">// else packet is DATA packet.</span></div>
<div class="line"><a id="l00724" name="l00724"></a><span class="lineno">  724</span> </div>
<div class="line"><a id="l00725" name="l00725"></a><span class="lineno">  725</span>  <span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow.html#a9d9cc2eeb10d398cff5591d446b763b8">Fine_time_pt</a> now = Fine_clock::now();</div>
<div class="line"><a id="l00726" name="l00726"></a><span class="lineno">  726</span>  <span class="keywordflow">if</span> ((pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#af1e8f11a36db3b967d9a5f61b115750a">m_slice_start</a> == <a class="code hl_typedef" href="namespaceflow.html#a9d9cc2eeb10d398cff5591d446b763b8">Fine_time_pt</a>()) || (now &gt;= (pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#af1e8f11a36db3b967d9a5f61b115750a">m_slice_start</a> + pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#af0c881081c6706dba393ef814386b143">m_slice_period</a>)))</div>
<div class="line"><a id="l00727" name="l00727"></a><span class="lineno">  727</span>  {</div>
<div class="line"><a id="l00728" name="l00728"></a><span class="lineno">  728</span>    <span class="comment">/* We are past the current time slice (if there is such a thing) and have a packet to send.  By</span></div>
<div class="line"><a id="l00729" name="l00729"></a><span class="lineno">  729</span><span class="comment">     * the algorithm in struct Send_pacing_data doc header, this means we create a new time slice with</span></div>
<div class="line"><a id="l00730" name="l00730"></a><span class="lineno">  730</span><span class="comment">     * starting time = now. */</span></div>
<div class="line"><a id="l00731" name="l00731"></a><span class="lineno">  731</span> </div>
<div class="line"><a id="l00732" name="l00732"></a><span class="lineno">  732</span>    <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Pacing: On [&quot;</span> &lt;&lt; sock &lt;&lt; <span class="stringliteral">&quot;]: &quot;</span></div>
<div class="line"><a id="l00733" name="l00733"></a><span class="lineno">  733</span>                   <span class="stringliteral">&quot;current time &quot;</span></div>
<div class="line"><a id="l00734" name="l00734"></a><span class="lineno">  734</span>                   <span class="stringliteral">&quot;slice [epoch+&quot;</span> &lt;&lt; duration_cast&lt;microseconds&gt;(pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#af1e8f11a36db3b967d9a5f61b115750a">m_slice_start</a>.time_since_epoch()) &lt;&lt; <span class="stringliteral">&quot; &quot;</span></div>
<div class="line"><a id="l00735" name="l00735"></a><span class="lineno">  735</span>                   <span class="stringliteral">&quot;over &quot;</span> &lt;&lt; duration_cast&lt;microseconds&gt;(pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#af0c881081c6706dba393ef814386b143">m_slice_period</a>) &lt;&lt; <span class="stringliteral">&quot;] is over.&quot;</span>);</div>
<div class="line"><a id="l00736" name="l00736"></a><span class="lineno">  736</span> </div>
<div class="line"><a id="l00737" name="l00737"></a><span class="lineno">  737</span>    <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#afb88dafa8ec5c1b412de86dfdccd9c8f">sock_pacing_new_time_slice</a>(sock, now);</div>
<div class="line"><a id="l00738" name="l00738"></a><span class="lineno">  738</span>    <span class="comment">/* pacing.{m_slice_start, m_slice_period, m_bytes_allowed_this_slice} have all been recomputed.</span></div>
<div class="line"><a id="l00739" name="l00739"></a><span class="lineno">  739</span><span class="comment">     * By definition &quot;now&quot; is in [m_slice_start, m_slice_start + m_slice_period), since m_slice_start</span></div>
<div class="line"><a id="l00740" name="l00740"></a><span class="lineno">  740</span><span class="comment">     * IS now (plus some epsilon). */</span></div>
<div class="line"><a id="l00741" name="l00741"></a><span class="lineno">  741</span>  }</div>
<div class="line"><a id="l00742" name="l00742"></a><span class="lineno">  742</span>  <span class="comment">// else if (&quot;now&quot; is basically in time range [m_slice_start, m_slice_start + m_slice_period)) {}</span></div>
<div class="line"><a id="l00743" name="l00743"></a><span class="lineno">  743</span> </div>
<div class="line"><a id="l00744" name="l00744"></a><span class="lineno">  744</span>  <span class="comment">/* We are in the current time slice.  The one just-queued packet can be sent if there is</span></div>
<div class="line"><a id="l00745" name="l00745"></a><span class="lineno">  745</span><span class="comment">   * m_bytes_allowed_this_slice budget; otherwise the timer must be started to fire at slice</span></div>
<div class="line"><a id="l00746" name="l00746"></a><span class="lineno">  746</span><span class="comment">   * end, at which point we&#39;re cleared to send the packet.  sock_pacing_process_q() performs this</span></div>
<div class="line"><a id="l00747" name="l00747"></a><span class="lineno">  747</span><span class="comment">   * (it is generalized to work with multiple packets on the queue, but it will work fine with just</span></div>
<div class="line"><a id="l00748" name="l00748"></a><span class="lineno">  748</span><span class="comment">   * one also).</span></div>
<div class="line"><a id="l00749" name="l00749"></a><span class="lineno">  749</span><span class="comment">   *</span></div>
<div class="line"><a id="l00750" name="l00750"></a><span class="lineno">  750</span><span class="comment">   * Note: If we just created the new time slice above, then m_bytes_allowed_this_slice &gt;=</span></div>
<div class="line"><a id="l00751" name="l00751"></a><span class="lineno">  751</span><span class="comment">   * max_block_size(), so certainly the following statement will immediately send the just-queued</span></div>
<div class="line"><a id="l00752" name="l00752"></a><span class="lineno">  752</span><span class="comment">   * packet. If the time slice was in progress, then it depends. */</span></div>
<div class="line"><a id="l00753" name="l00753"></a><span class="lineno">  753</span> </div>
<div class="line"><a id="l00754" name="l00754"></a><span class="lineno">  754</span>  <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a73cfaa8e5ed7fe845f759a30423a9cbf">sock_pacing_process_q</a>(sock, <span class="keyword">false</span>);</div>
<div class="line"><a id="l00755" name="l00755"></a><span class="lineno">  755</span>} <span class="comment">// Node::sock_pacing_new_packet_ready()</span></div>
<div class="line"><a id="l00756" name="l00756"></a><span class="lineno">  756</span> </div>
<div class="line"><a id="l00757" name="l00757"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#afb88dafa8ec5c1b412de86dfdccd9c8f">  757</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#afb88dafa8ec5c1b412de86dfdccd9c8f">Node::sock_pacing_new_time_slice</a>(<a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Peer_socket::Ptr</a> sock, <span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow.html#a9d9cc2eeb10d398cff5591d446b763b8">Fine_time_pt</a>&amp; now)</div>
<div class="line"><a id="l00758" name="l00758"></a><span class="lineno">  758</span>{</div>
<div class="line"><a id="l00759" name="l00759"></a><span class="lineno">  759</span>  <span class="keyword">using </span>boost::chrono::duration_cast;</div>
<div class="line"><a id="l00760" name="l00760"></a><span class="lineno">  760</span>  <span class="keyword">using </span>boost::chrono::microseconds;</div>
<div class="line"><a id="l00761" name="l00761"></a><span class="lineno">  761</span>  <span class="keyword">using </span>boost::chrono::milliseconds;</div>
<div class="line"><a id="l00762" name="l00762"></a><span class="lineno">  762</span>  <span class="keyword">using </span>std::max;</div>
<div class="line"><a id="l00763" name="l00763"></a><span class="lineno">  763</span> </div>
<div class="line"><a id="l00764" name="l00764"></a><span class="lineno">  764</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00765" name="l00765"></a><span class="lineno">  765</span> </div>
<div class="line"><a id="l00766" name="l00766"></a><span class="lineno">  766</span>  <span class="comment">// For brevity and a bit of speed.</span></div>
<div class="line"><a id="l00767" name="l00767"></a><span class="lineno">  767</span>  <a class="code hl_struct" href="structflow_1_1net__flow_1_1Send__pacing__data.html">Send_pacing_data</a>&amp; pacing = sock-&gt;m_snd_pacing_data;</div>
<div class="line"><a id="l00768" name="l00768"></a><span class="lineno">  768</span>  <span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow.html#a48799f1263cdeedec125be51a3db2b79">Fine_duration</a>&amp; srtt = sock-&gt;m_snd_smoothed_round_trip_time;</div>
<div class="line"><a id="l00769" name="l00769"></a><span class="lineno">  769</span> </div>
<div class="line"><a id="l00770" name="l00770"></a><span class="lineno">  770</span>  assert(srtt != Fine_duration::zero());</div>
<div class="line"><a id="l00771" name="l00771"></a><span class="lineno">  771</span> </div>
<div class="line"><a id="l00772" name="l00772"></a><span class="lineno">  772</span>  <span class="comment">// New slice starts now.</span></div>
<div class="line"><a id="l00773" name="l00773"></a><span class="lineno">  773</span>  pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#af1e8f11a36db3b967d9a5f61b115750a">m_slice_start</a> = now;</div>
<div class="line"><a id="l00774" name="l00774"></a><span class="lineno">  774</span> </div>
<div class="line"><a id="l00775" name="l00775"></a><span class="lineno">  775</span>  <span class="comment">/* Per struct Send_pacing_data doc header: If we had perfectly fine timer precision, then we&#39;d</span></div>
<div class="line"><a id="l00776" name="l00776"></a><span class="lineno">  776</span><span class="comment">   * want a slice that is SRTT / CWND.  CWND = (congestion window in bytes / max-block-size).</span></div>
<div class="line"><a id="l00777" name="l00777"></a><span class="lineno">  777</span><span class="comment">   * To minimize truncation error, then, it is X / (Y / Z) = X * Z / Y, where X, Y, Z are SRTT,</span></div>
<div class="line"><a id="l00778" name="l00778"></a><span class="lineno">  778</span><span class="comment">   * max-block-size, and congestion window in bytes, respectively. */</span></div>
<div class="line"><a id="l00779" name="l00779"></a><span class="lineno">  779</span>  <a class="code hl_typedef" href="namespaceflow.html#a48799f1263cdeedec125be51a3db2b79">Fine_duration</a> slice_ideal_period</div>
<div class="line"><a id="l00780" name="l00780"></a><span class="lineno">  780</span>    = srtt * sock-&gt;max_block_size() / sock-&gt;m_snd_cong_ctl-&gt;congestion_window_bytes();</div>
<div class="line"><a id="l00781" name="l00781"></a><span class="lineno">  781</span>  <span class="keywordflow">if</span> (slice_ideal_period == Fine_duration::zero())</div>
<div class="line"><a id="l00782" name="l00782"></a><span class="lineno">  782</span>  {</div>
<div class="line"><a id="l00783" name="l00783"></a><span class="lineno">  783</span>    <span class="comment">// Avoid division by zero and any other tomfoolery below....</span></div>
<div class="line"><a id="l00784" name="l00784"></a><span class="lineno">  784</span>    slice_ideal_period = <a class="code hl_typedef" href="namespaceflow.html#a48799f1263cdeedec125be51a3db2b79">Fine_duration</a>(1);</div>
<div class="line"><a id="l00785" name="l00785"></a><span class="lineno">  785</span>  }</div>
<div class="line"><a id="l00786" name="l00786"></a><span class="lineno">  786</span> </div>
<div class="line"><a id="l00787" name="l00787"></a><span class="lineno">  787</span>  <a class="code hl_typedef" href="namespaceflow.html#a48799f1263cdeedec125be51a3db2b79">Fine_duration</a> timer_min_period = <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a43c42121d48a55a43a48e3a84bd10595">opt</a>(<a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#ade08f0457943f803c23b55649139c0d3">m_opts</a>.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Node__options.html#abef8889be34024b07eff3211a3213d5f">m_st_timer_min_period</a>);</div>
<div class="line"><a id="l00788" name="l00788"></a><span class="lineno">  788</span>  <span class="keywordflow">if</span> (timer_min_period == Fine_duration::zero())</div>
<div class="line"><a id="l00789" name="l00789"></a><span class="lineno">  789</span>  {</div>
<div class="line"><a id="l00790" name="l00790"></a><span class="lineno">  790</span>    <span class="comment">/* They want us to pick the a nice upper bound on the timer precision ourselves.</span></div>
<div class="line"><a id="l00791" name="l00791"></a><span class="lineno">  791</span><span class="comment">     *</span></div>
<div class="line"><a id="l00792" name="l00792"></a><span class="lineno">  792</span><span class="comment">     * This is conservative; in my tests Windows seem to have the worst timer precision, and it is</span></div>
<div class="line"><a id="l00793" name="l00793"></a><span class="lineno">  793</span><span class="comment">     * about 15 msec. @todo Perhaps choose here based on platform. It can get hairy, as there is</span></div>
<div class="line"><a id="l00794" name="l00794"></a><span class="lineno">  794</span><span class="comment">     * wide variation, so it would require much experimentation; but might be worth it for</span></div>
<div class="line"><a id="l00795" name="l00795"></a><span class="lineno">  795</span><span class="comment">     * performance. */</span></div>
<div class="line"><a id="l00796" name="l00796"></a><span class="lineno">  796</span>    <span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow.html#a48799f1263cdeedec125be51a3db2b79">Fine_duration</a> TIMER_MIN_PERIOD_DEFAULT = milliseconds(15);</div>
<div class="line"><a id="l00797" name="l00797"></a><span class="lineno">  797</span>    timer_min_period = TIMER_MIN_PERIOD_DEFAULT;</div>
<div class="line"><a id="l00798" name="l00798"></a><span class="lineno">  798</span>  }</div>
<div class="line"><a id="l00799" name="l00799"></a><span class="lineno">  799</span> </div>
<div class="line"><a id="l00800" name="l00800"></a><span class="lineno">  800</span>  <span class="comment">/* Per Send_pacing_data doc header, the actual slice period is slice_ideal_period, unless that</span></div>
<div class="line"><a id="l00801" name="l00801"></a><span class="lineno">  801</span><span class="comment">   * is below the timer&#39;s capabilities; in which case it is timer_min_period. */</span></div>
<div class="line"><a id="l00802" name="l00802"></a><span class="lineno">  802</span>  pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#af0c881081c6706dba393ef814386b143">m_slice_period</a> = max(slice_ideal_period, timer_min_period);</div>
<div class="line"><a id="l00803" name="l00803"></a><span class="lineno">  803</span> </div>
<div class="line"><a id="l00804" name="l00804"></a><span class="lineno">  804</span>  <span class="comment">/* Finally, allow (in this time slice) a burst of as few full-sized blocks as possible while</span></div>
<div class="line"><a id="l00805" name="l00805"></a><span class="lineno">  805</span><span class="comment">   * staying below the target SRTT / CWND rate: floor(m_slice_period / slice_ideal_period).</span></div>
<div class="line"><a id="l00806" name="l00806"></a><span class="lineno">  806</span><span class="comment">   * Note that when slice_ideal_period &gt;= timer_min_period, i.e. the timer precision is fine enough</span></div>
<div class="line"><a id="l00807" name="l00807"></a><span class="lineno">  807</span><span class="comment">   * to handle the desired rate exactly, then this value will always equal 1.</span></div>
<div class="line"><a id="l00808" name="l00808"></a><span class="lineno">  808</span><span class="comment">   *</span></div>
<div class="line"><a id="l00809" name="l00809"></a><span class="lineno">  809</span><span class="comment">   * Also, convert to bytes when actually assigning the data member.</span></div>
<div class="line"><a id="l00810" name="l00810"></a><span class="lineno">  810</span><span class="comment">   *</span></div>
<div class="line"><a id="l00811" name="l00811"></a><span class="lineno">  811</span><span class="comment">   * @todo Consider doing some kind of rounding instead of using floor(). */</span></div>
<div class="line"><a id="l00812" name="l00812"></a><span class="lineno">  812</span>  pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#a9cfd632911c3c4acd8e048dae28ded46">m_bytes_allowed_this_slice</a></div>
<div class="line"><a id="l00813" name="l00813"></a><span class="lineno">  813</span>    = <span class="keyword">static_cast&lt;</span><span class="keywordtype">size_t</span><span class="keyword">&gt;</span>(pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#af0c881081c6706dba393ef814386b143">m_slice_period</a> * sock-&gt;max_block_size() / slice_ideal_period);</div>
<div class="line"><a id="l00814" name="l00814"></a><span class="lineno">  814</span> </div>
<div class="line"><a id="l00815" name="l00815"></a><span class="lineno">  815</span>  <span class="comment">/* If I just use the above math, I notice that over time the queue size can drift becoming slowly</span></div>
<div class="line"><a id="l00816" name="l00816"></a><span class="lineno">  816</span><span class="comment">   * larger and larger as more and more time slices go by. I believe it&#39;s due to our floor math</span></div>
<div class="line"><a id="l00817" name="l00817"></a><span class="lineno">  817</span><span class="comment">   * above, but I have not yet fully investigated it. @todo Investigate it.</span></div>
<div class="line"><a id="l00818" name="l00818"></a><span class="lineno">  818</span><span class="comment">   *</span></div>
<div class="line"><a id="l00819" name="l00819"></a><span class="lineno">  819</span><span class="comment">   * However if I just increase the number of bytes allowed per slice a little bit, it makes the</span></div>
<div class="line"><a id="l00820" name="l00820"></a><span class="lineno">  820</span><span class="comment">   * drift go away and probably doesn&#39;t reduce the effectiveness of the pacing much. */</span></div>
<div class="line"><a id="l00821" name="l00821"></a><span class="lineno">  821</span>  <span class="keyword">const</span> <span class="keywordtype">size_t</span> QUEUE_SIZE_DRIFT_PREVENTION_PCT = 110; <span class="comment">// @todo Maybe make it a configurable option.</span></div>
<div class="line"><a id="l00822" name="l00822"></a><span class="lineno">  822</span>  pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#a9cfd632911c3c4acd8e048dae28ded46">m_bytes_allowed_this_slice</a> *= QUEUE_SIZE_DRIFT_PREVENTION_PCT;</div>
<div class="line"><a id="l00823" name="l00823"></a><span class="lineno">  823</span>  pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#a9cfd632911c3c4acd8e048dae28ded46">m_bytes_allowed_this_slice</a> /= 100;</div>
<div class="line"><a id="l00824" name="l00824"></a><span class="lineno">  824</span> </div>
<div class="line"><a id="l00825" name="l00825"></a><span class="lineno">  825</span>  assert(pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#a9cfd632911c3c4acd8e048dae28ded46">m_bytes_allowed_this_slice</a> &gt;= sock-&gt;max_block_size());</div>
<div class="line"><a id="l00826" name="l00826"></a><span class="lineno">  826</span> </div>
<div class="line"><a id="l00827" name="l00827"></a><span class="lineno">  827</span>  <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Pacing: On [&quot;</span> &lt;&lt; sock &lt;&lt; <span class="stringliteral">&quot;]: &quot;</span></div>
<div class="line"><a id="l00828" name="l00828"></a><span class="lineno">  828</span>                 <span class="stringliteral">&quot;new time &quot;</span></div>
<div class="line"><a id="l00829" name="l00829"></a><span class="lineno">  829</span>                 <span class="stringliteral">&quot;slice [epoch+&quot;</span> &lt;&lt; duration_cast&lt;microseconds&gt;(pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#af1e8f11a36db3b967d9a5f61b115750a">m_slice_start</a>.time_since_epoch()) &lt;&lt; <span class="stringliteral">&quot; &quot;</span></div>
<div class="line"><a id="l00830" name="l00830"></a><span class="lineno">  830</span>                 <span class="stringliteral">&quot;over &quot;</span> &lt;&lt; duration_cast&lt;microseconds&gt;(pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#af0c881081c6706dba393ef814386b143">m_slice_period</a>) &lt;&lt; <span class="stringliteral">&quot;]; &quot;</span></div>
<div class="line"><a id="l00831" name="l00831"></a><span class="lineno">  831</span>                 <span class="stringliteral">&quot;ideal slice period = [SRTT &quot;</span> &lt;&lt; duration_cast&lt;microseconds&gt;(srtt) &lt;&lt; <span class="stringliteral">&quot;] / &quot;</span></div>
<div class="line"><a id="l00832" name="l00832"></a><span class="lineno">  832</span>                 <span class="stringliteral">&quot;([cong_wnd &quot;</span> &lt;&lt; sock-&gt;m_snd_cong_ctl-&gt;congestion_window_bytes() &lt;&lt; <span class="stringliteral">&quot;] / &quot;</span></div>
<div class="line"><a id="l00833" name="l00833"></a><span class="lineno">  833</span>                 <span class="stringliteral">&quot;[max-block-size &quot;</span> &lt;&lt; sock-&gt;max_block_size() &lt;&lt; <span class="stringliteral">&quot;]) = &quot;</span></div>
<div class="line"><a id="l00834" name="l00834"></a><span class="lineno">  834</span>                 <span class="stringliteral">&quot;[&quot;</span> &lt;&lt; duration_cast&lt;microseconds&gt;(slice_ideal_period) &lt;&lt; <span class="stringliteral">&quot;]; &quot;</span></div>
<div class="line"><a id="l00835" name="l00835"></a><span class="lineno">  835</span>                 <span class="stringliteral">&quot;timer_min_period = [&quot;</span> &lt;&lt; duration_cast&lt;microseconds&gt;(timer_min_period) &lt;&lt; <span class="stringliteral">&quot;]; &quot;</span></div>
<div class="line"><a id="l00836" name="l00836"></a><span class="lineno">  836</span>                 <span class="stringliteral">&quot;bytes_allowed = max(ideal, min) / ideal * max-block-size * &quot;</span></div>
<div class="line"><a id="l00837" name="l00837"></a><span class="lineno">  837</span>                 <span class="stringliteral">&quot;[&quot;</span> &lt;&lt; QUEUE_SIZE_DRIFT_PREVENTION_PCT &lt;&lt; <span class="stringliteral">&quot;%] = &quot;</span></div>
<div class="line"><a id="l00838" name="l00838"></a><span class="lineno">  838</span>                 <span class="stringliteral">&quot;[&quot;</span> &lt;&lt; pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#a9cfd632911c3c4acd8e048dae28ded46">m_bytes_allowed_this_slice</a> &lt;&lt; <span class="stringliteral">&quot;].&quot;</span>);</div>
<div class="line"><a id="l00839" name="l00839"></a><span class="lineno">  839</span>} <span class="comment">// Node::sock_pacing_new_time_slice()</span></div>
<div class="line"><a id="l00840" name="l00840"></a><span class="lineno">  840</span> </div>
<div class="line"><a id="l00841" name="l00841"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#a73cfaa8e5ed7fe845f759a30423a9cbf">  841</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a73cfaa8e5ed7fe845f759a30423a9cbf">Node::sock_pacing_process_q</a>(<a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Peer_socket::Ptr</a> sock, <span class="keywordtype">bool</span> executing_after_delay)</div>
<div class="line"><a id="l00842" name="l00842"></a><span class="lineno">  842</span>{</div>
<div class="line"><a id="l00843" name="l00843"></a><span class="lineno">  843</span>  <span class="keyword">using </span>boost::chrono::milliseconds;</div>
<div class="line"><a id="l00844" name="l00844"></a><span class="lineno">  844</span>  <span class="keyword">using </span>boost::chrono::round;</div>
<div class="line"><a id="l00845" name="l00845"></a><span class="lineno">  845</span>  <span class="keyword">using </span>boost::shared_ptr;</div>
<div class="line"><a id="l00846" name="l00846"></a><span class="lineno">  846</span>  <span class="keyword">using </span>boost::weak_ptr;</div>
<div class="line"><a id="l00847" name="l00847"></a><span class="lineno">  847</span>  <span class="keyword">using </span>boost::static_pointer_cast;</div>
<div class="line"><a id="l00848" name="l00848"></a><span class="lineno">  848</span>  <span class="keyword">using </span>boost::dynamic_pointer_cast;</div>
<div class="line"><a id="l00849" name="l00849"></a><span class="lineno">  849</span>  <span class="keyword">using </span>std::max;</div>
<div class="line"><a id="l00850" name="l00850"></a><span class="lineno">  850</span> </div>
<div class="line"><a id="l00851" name="l00851"></a><span class="lineno">  851</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00852" name="l00852"></a><span class="lineno">  852</span> </div>
<div class="line"><a id="l00853" name="l00853"></a><span class="lineno">  853</span>  <span class="comment">/* Pre-condition is that the current time is within the current time slice, and that all other</span></div>
<div class="line"><a id="l00854" name="l00854"></a><span class="lineno">  854</span><span class="comment">   * invariants hold (including that the head packet, if any, is a DATA packet).  So now we send as</span></div>
<div class="line"><a id="l00855" name="l00855"></a><span class="lineno">  855</span><span class="comment">   * many packets as still allowed by the budget in m_bytes_allowed_this_slice.  If anything remains</span></div>
<div class="line"><a id="l00856" name="l00856"></a><span class="lineno">  856</span><span class="comment">   * beyond that, we schedule a timer to hit at the end of the time slice to get the rest. */</span></div>
<div class="line"><a id="l00857" name="l00857"></a><span class="lineno">  857</span> </div>
<div class="line"><a id="l00858" name="l00858"></a><span class="lineno">  858</span>  <span class="comment">// For brevity and a bit of speed.</span></div>
<div class="line"><a id="l00859" name="l00859"></a><span class="lineno">  859</span>  <a class="code hl_struct" href="structflow_1_1net__flow_1_1Send__pacing__data.html">Send_pacing_data</a>&amp; pacing = sock-&gt;m_snd_pacing_data;</div>
<div class="line"><a id="l00860" name="l00860"></a><span class="lineno">  860</span>  shared_ptr&lt;const Data_packet&gt; head_packet;</div>
<div class="line"><a id="l00861" name="l00861"></a><span class="lineno">  861</span> </div>
<div class="line"><a id="l00862" name="l00862"></a><span class="lineno">  862</span>  <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Pacing: On [&quot;</span> &lt;&lt; sock &lt;&lt; <span class="stringliteral">&quot;]: processing queue; queue size [&quot;</span> &lt;&lt; pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.size() &lt;&lt; <span class="stringliteral">&quot;]; &quot;</span></div>
<div class="line"><a id="l00863" name="l00863"></a><span class="lineno">  863</span>                 <span class="stringliteral">&quot;byte budget [&quot;</span> &lt;&lt; sock-&gt;bytes_blocks_str(pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#a9cfd632911c3c4acd8e048dae28ded46">m_bytes_allowed_this_slice</a>) &lt;&lt; <span class="stringliteral">&quot;] remaining in this &quot;</span></div>
<div class="line"><a id="l00864" name="l00864"></a><span class="lineno">  864</span>                 <span class="stringliteral">&quot;slice.&quot;</span>);</div>
<div class="line"><a id="l00865" name="l00865"></a><span class="lineno">  865</span> </div>
<div class="line"><a id="l00866" name="l00866"></a><span class="lineno">  866</span>  <span class="comment">/* Pop things from queue until we&#39;ve run out of pacing byte budget for this time slice, or until</span></div>
<div class="line"><a id="l00867" name="l00867"></a><span class="lineno">  867</span><span class="comment">   * there is nothing left to send. */</span></div>
<div class="line"><a id="l00868" name="l00868"></a><span class="lineno">  868</span>  <span class="keywordflow">while</span> ((!pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.empty())</div>
<div class="line"><a id="l00869" name="l00869"></a><span class="lineno">  869</span>         &amp;&amp; (pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#a9cfd632911c3c4acd8e048dae28ded46">m_bytes_allowed_this_slice</a></div>
<div class="line"><a id="l00870" name="l00870"></a><span class="lineno">  870</span>             &gt;= (head_packet = static_pointer_cast&lt;const Data_packet&gt;(pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.front()))-&gt;m_data.size()))</div>
<div class="line"><a id="l00871" name="l00871"></a><span class="lineno">  871</span>  {</div>
<div class="line"><a id="l00872" name="l00872"></a><span class="lineno">  872</span>    <span class="comment">// Explicit invariant: header_packet is DATA.  We always send non-DATA packets as soon as they get to head of queue.</span></div>
<div class="line"><a id="l00873" name="l00873"></a><span class="lineno">  873</span> </div>
<div class="line"><a id="l00874" name="l00874"></a><span class="lineno">  874</span>    <span class="comment">// It is a DATA packet at head of queue, and there is enough pacing budget to send it now.</span></div>
<div class="line"><a id="l00875" name="l00875"></a><span class="lineno">  875</span> </div>
<div class="line"><a id="l00876" name="l00876"></a><span class="lineno">  876</span>    <span class="comment">// Use up the budget.</span></div>
<div class="line"><a id="l00877" name="l00877"></a><span class="lineno">  877</span>    pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#a9cfd632911c3c4acd8e048dae28ded46">m_bytes_allowed_this_slice</a> -= head_packet-&gt;m_data.size();</div>
<div class="line"><a id="l00878" name="l00878"></a><span class="lineno">  878</span> </div>
<div class="line"><a id="l00879" name="l00879"></a><span class="lineno">  879</span>    <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Will send [&quot;</span> &lt;&lt; head_packet-&gt;m_data.size() &lt;&lt; <span class="stringliteral">&quot;] bytes of data; budget now &quot;</span></div>
<div class="line"><a id="l00880" name="l00880"></a><span class="lineno">  880</span>                   <span class="stringliteral">&quot;[&quot;</span> &lt;&lt; sock-&gt;bytes_blocks_str(pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#a9cfd632911c3c4acd8e048dae28ded46">m_bytes_allowed_this_slice</a>) &lt;&lt; <span class="stringliteral">&quot;]; &quot;</span></div>
<div class="line"><a id="l00881" name="l00881"></a><span class="lineno">  881</span>                   <span class="stringliteral">&quot;queue size now [&quot;</span> &lt;&lt; (pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.size() - 1) &lt;&lt; <span class="stringliteral">&quot;].&quot;</span>);</div>
<div class="line"><a id="l00882" name="l00882"></a><span class="lineno">  882</span> </div>
<div class="line"><a id="l00883" name="l00883"></a><span class="lineno">  883</span>    <span class="comment">// Send it.  executing_after_delay &lt;=&gt; packet being sent was delayed by pacing as opposed to sent immediately.</span></div>
<div class="line"><a id="l00884" name="l00884"></a><span class="lineno">  884</span>    <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a0db2bb587e0407077b11368df075f0ff">async_sock_low_lvl_packet_send</a>(sock, std::move(head_packet), executing_after_delay);</div>
<div class="line"><a id="l00885" name="l00885"></a><span class="lineno">  885</span> </div>
<div class="line"><a id="l00886" name="l00886"></a><span class="lineno">  886</span>    pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.pop_front(); <span class="comment">// After this the raw pointer in head_packet should be freed.</span></div>
<div class="line"><a id="l00887" name="l00887"></a><span class="lineno">  887</span> </div>
<div class="line"><a id="l00888" name="l00888"></a><span class="lineno">  888</span>    <span class="comment">/* Since we&#39;ve popped a packet, another packet is now at the head.  We must maintain the</span></div>
<div class="line"><a id="l00889" name="l00889"></a><span class="lineno">  889</span><span class="comment">     * invariant that no non-DATA packet is at the head of the queue (see struct Send_pacing_data</span></div>
<div class="line"><a id="l00890" name="l00890"></a><span class="lineno">  890</span><span class="comment">     * doc header for reasoning), i.e. we should send any such packets immediately.  Do so until</span></div>
<div class="line"><a id="l00891" name="l00891"></a><span class="lineno">  891</span><span class="comment">     * we run out or encounter a DATA packet. */</span></div>
<div class="line"><a id="l00892" name="l00892"></a><span class="lineno">  892</span> </div>
<div class="line"><a id="l00893" name="l00893"></a><span class="lineno">  893</span>    <span class="comment">// Subtlety: Using dynamic_pointer_cast&lt;&gt; instead of typeid() to check packet type to avoid &quot;side effect&quot; warning.</span></div>
<div class="line"><a id="l00894" name="l00894"></a><span class="lineno">  894</span>    <a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aef7998db71c60eeb5d1e3d1a97c14886">Low_lvl_packet::Const_ptr</a> head_packet_base;</div>
<div class="line"><a id="l00895" name="l00895"></a><span class="lineno">  895</span>    <span class="keywordflow">while</span> ((!pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.empty())</div>
<div class="line"><a id="l00896" name="l00896"></a><span class="lineno">  896</span>           &amp;&amp; (!(dynamic_pointer_cast&lt;const Data_packet&gt;(head_packet_base = pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.front()))))</div>
<div class="line"><a id="l00897" name="l00897"></a><span class="lineno">  897</span>    {</div>
<div class="line"><a id="l00898" name="l00898"></a><span class="lineno">  898</span>      <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Pacing: On [&quot;</span> &lt;&lt; sock &lt;&lt; <span class="stringliteral">&quot;]: due to packet type, sending immediately since at head of queue; &quot;</span></div>
<div class="line"><a id="l00899" name="l00899"></a><span class="lineno">  899</span>                     <span class="stringliteral">&quot;queue size now [&quot;</span> &lt;&lt; (pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.size() - 1) &lt;&lt; <span class="stringliteral">&quot;].&quot;</span>);</div>
<div class="line"><a id="l00900" name="l00900"></a><span class="lineno">  900</span> </div>
<div class="line"><a id="l00901" name="l00901"></a><span class="lineno">  901</span>      <span class="comment">// See above cmnt about last arg.</span></div>
<div class="line"><a id="l00902" name="l00902"></a><span class="lineno">  902</span>      <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a0db2bb587e0407077b11368df075f0ff">async_sock_low_lvl_packet_send</a>(sock, std::move(head_packet_base), executing_after_delay);</div>
<div class="line"><a id="l00903" name="l00903"></a><span class="lineno">  903</span> </div>
<div class="line"><a id="l00904" name="l00904"></a><span class="lineno">  904</span>      pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.pop_front(); <span class="comment">// After this the raw pointer in head_packet should be freed.</span></div>
<div class="line"><a id="l00905" name="l00905"></a><span class="lineno">  905</span> </div>
<div class="line"><a id="l00906" name="l00906"></a><span class="lineno">  906</span>      <span class="comment">// Note that, as discussed in struct Send_pacing_data doc header, a non-DATA packet is worth 0 budget.</span></div>
<div class="line"><a id="l00907" name="l00907"></a><span class="lineno">  907</span>    }</div>
<div class="line"><a id="l00908" name="l00908"></a><span class="lineno">  908</span>  } <span class="comment">// while ((m_packet_q not empty) &amp;&amp; (more m_bytes_allowed_this_slice budget available))</span></div>
<div class="line"><a id="l00909" name="l00909"></a><span class="lineno">  909</span> </div>
<div class="line"><a id="l00910" name="l00910"></a><span class="lineno">  910</span>  <span class="keywordflow">if</span> (pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.empty())</div>
<div class="line"><a id="l00911" name="l00911"></a><span class="lineno">  911</span>  {</div>
<div class="line"><a id="l00912" name="l00912"></a><span class="lineno">  912</span>    <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Pacing: Queue emptied.&quot;</span>);</div>
<div class="line"><a id="l00913" name="l00913"></a><span class="lineno">  913</span> </div>
<div class="line"><a id="l00914" name="l00914"></a><span class="lineno">  914</span>    <span class="comment">// Successfully sent off entire queue.  Pacing done for now -- until the next sock_pacing_new_packet_ready().</span></div>
<div class="line"><a id="l00915" name="l00915"></a><span class="lineno">  915</span>    <span class="keywordflow">return</span>;</div>
<div class="line"><a id="l00916" name="l00916"></a><span class="lineno">  916</span>  }</div>
<div class="line"><a id="l00917" name="l00917"></a><span class="lineno">  917</span>  <span class="comment">// else</span></div>
<div class="line"><a id="l00918" name="l00918"></a><span class="lineno">  918</span> </div>
<div class="line"><a id="l00919" name="l00919"></a><span class="lineno">  919</span>  <span class="comment">/* No more budget left in this pacing time slice, but there is at least one packet in the queue</span></div>
<div class="line"><a id="l00920" name="l00920"></a><span class="lineno">  920</span><span class="comment">   * still.  Per algorithm, a fresh slice should begin ASAP (since packets must be sent ASAP but no</span></div>
<div class="line"><a id="l00921" name="l00921"></a><span class="lineno">  921</span><span class="comment">   * sooner).  Therefore schedule timer for the end of the time slice, which is just</span></div>
<div class="line"><a id="l00922" name="l00922"></a><span class="lineno">  922</span><span class="comment">   * m_slice_start + m_slice_period. */</span></div>
<div class="line"><a id="l00923" name="l00923"></a><span class="lineno">  923</span>  <span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow.html#a9d9cc2eeb10d398cff5591d446b763b8">Fine_time_pt</a> slice_end = pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#af1e8f11a36db3b967d9a5f61b115750a">m_slice_start</a> + pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#af0c881081c6706dba393ef814386b143">m_slice_period</a>;</div>
<div class="line"><a id="l00924" name="l00924"></a><span class="lineno">  924</span> </div>
<div class="line"><a id="l00925" name="l00925"></a><span class="lineno">  925</span>  pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#a100a406213a5edf2854440397719bb91">m_slice_timer</a>.expires_at(slice_end);</div>
<div class="line"><a id="l00926" name="l00926"></a><span class="lineno">  926</span>  <span class="comment">// (Even if slice_end is slightly in the past, that&#39;ll just mean it&#39;ll fire ASAP.)</span></div>
<div class="line"><a id="l00927" name="l00927"></a><span class="lineno">  927</span> </div>
<div class="line"><a id="l00928" name="l00928"></a><span class="lineno">  928</span>  <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Pacing: Exhausted budget; queue size [&quot;</span> &lt;&lt; pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">m_packet_q</a>.size() &lt;&lt; <span class="stringliteral">&quot;]; &quot;</span></div>
<div class="line"><a id="l00929" name="l00929"></a><span class="lineno">  929</span>                 <span class="stringliteral">&quot;scheduling next processing at end of time slice &quot;</span></div>
<div class="line"><a id="l00930" name="l00930"></a><span class="lineno">  930</span>                 <span class="stringliteral">&quot;in [&quot;</span> &lt;&lt; round&lt;milliseconds&gt;(slice_end - Fine_clock::now()) &lt;&lt; <span class="stringliteral">&quot;].&quot;</span>);</div>
<div class="line"><a id="l00931" name="l00931"></a><span class="lineno">  931</span> </div>
<div class="line"><a id="l00932" name="l00932"></a><span class="lineno">  932</span>  <span class="comment">// When triggered or canceled, call this-&gt;sock_pacing_time_slice_end(sock, &lt;error code&gt;).</span></div>
<div class="line"><a id="l00933" name="l00933"></a><span class="lineno">  933</span>  pacing.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Send__pacing__data.html#a100a406213a5edf2854440397719bb91">m_slice_timer</a>.async_wait([<span class="keyword">this</span>, sock_observer = weak_ptr&lt;Peer_socket&gt;(sock)]</div>
<div class="line"><a id="l00934" name="l00934"></a><span class="lineno">  934</span>                                    (<span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow.html#a29eaaa9d0fac4ce87d8b969222dbed09">Error_code</a>&amp; sys_err_code)</div>
<div class="line"><a id="l00935" name="l00935"></a><span class="lineno">  935</span>  {</div>
<div class="line"><a id="l00936" name="l00936"></a><span class="lineno">  936</span>    <span class="keyword">auto</span> sock = sock_observer.lock();</div>
<div class="line"><a id="l00937" name="l00937"></a><span class="lineno">  937</span>    <span class="keywordflow">if</span> (sock)</div>
<div class="line"><a id="l00938" name="l00938"></a><span class="lineno">  938</span>    {</div>
<div class="line"><a id="l00939" name="l00939"></a><span class="lineno">  939</span>      <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a3fd57a0d025a78cbeccfd8a15a983575">sock_pacing_time_slice_end</a>(sock, sys_err_code);</div>
<div class="line"><a id="l00940" name="l00940"></a><span class="lineno">  940</span>    }</div>
<div class="line"><a id="l00941" name="l00941"></a><span class="lineno">  941</span>    <span class="comment">// else { Possible or not, allow for this possibility for maintainability. }</span></div>
<div class="line"><a id="l00942" name="l00942"></a><span class="lineno">  942</span>  });</div>
<div class="line"><a id="l00943" name="l00943"></a><span class="lineno">  943</span> </div>
<div class="line"><a id="l00944" name="l00944"></a><span class="lineno">  944</span>  <span class="comment">// More work to do later, but for now we&#39;ve been successful.</span></div>
<div class="line"><a id="l00945" name="l00945"></a><span class="lineno">  945</span> </div>
<div class="line"><a id="l00946" name="l00946"></a><span class="lineno">  946</span>  <span class="comment">/* That&#39;s it.  The only reason the timer would get canceled is if we go into CLOSED state, in</span></div>
<div class="line"><a id="l00947" name="l00947"></a><span class="lineno">  947</span><span class="comment">   * which case it can just do nothing. */</span></div>
<div class="line"><a id="l00948" name="l00948"></a><span class="lineno">  948</span>} <span class="comment">// Node::sock_pacing_process_q()</span></div>
<div class="line"><a id="l00949" name="l00949"></a><span class="lineno">  949</span> </div>
<div class="line"><a id="l00950" name="l00950"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#a3fd57a0d025a78cbeccfd8a15a983575">  950</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a3fd57a0d025a78cbeccfd8a15a983575">Node::sock_pacing_time_slice_end</a>(<a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Peer_socket::Ptr</a> sock, [[maybe_unused]] <span class="keyword">const</span> <a class="code hl_typedef" href="namespaceflow.html#a29eaaa9d0fac4ce87d8b969222dbed09">Error_code</a>&amp; sys_err_code)</div>
<div class="line"><a id="l00951" name="l00951"></a><span class="lineno">  951</span>{</div>
<div class="line"><a id="l00952" name="l00952"></a><span class="lineno">  952</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00953" name="l00953"></a><span class="lineno">  953</span> </div>
<div class="line"><a id="l00954" name="l00954"></a><span class="lineno">  954</span>  <span class="comment">// As always, no need to lock m_state, unless we plan to alter it, since no other thread can alter it.</span></div>
<div class="line"><a id="l00955" name="l00955"></a><span class="lineno">  955</span>  <span class="keywordflow">if</span> (sock-&gt;m_state == <a class="code hl_enumvalue" href="classflow_1_1net__flow_1_1Peer__socket.html#a6a9bb727c8982f24bcf978c10c2c7345a6abcbd1be8478aa03e76aec9c540e14f">Peer_socket::State::S_CLOSED</a>)</div>
<div class="line"><a id="l00956" name="l00956"></a><span class="lineno">  956</span>  {</div>
<div class="line"><a id="l00957" name="l00957"></a><span class="lineno">  957</span>    <span class="comment">/* Once state is CLOSED, the socket is dead -- all packets have been sent.  A corollary of</span></div>
<div class="line"><a id="l00958" name="l00958"></a><span class="lineno">  958</span><span class="comment">     * that is that if we must send an RST, then always send it immediately (even if it has to</span></div>
<div class="line"><a id="l00959" name="l00959"></a><span class="lineno">  959</span><span class="comment">     * jump ahead of other packets waiting on queue). */</span></div>
<div class="line"><a id="l00960" name="l00960"></a><span class="lineno">  960</span>    <span class="keywordflow">return</span>;</div>
<div class="line"><a id="l00961" name="l00961"></a><span class="lineno">  961</span>  }</div>
<div class="line"><a id="l00962" name="l00962"></a><span class="lineno">  962</span> </div>
<div class="line"><a id="l00963" name="l00963"></a><span class="lineno">  963</span>  <span class="comment">// We only cancel the timer when we close socket, and we&#39;ve already returned if we&#39;d done that.</span></div>
<div class="line"><a id="l00964" name="l00964"></a><span class="lineno">  964</span>  assert(sys_err_code != boost::asio::error::operation_aborted);</div>
<div class="line"><a id="l00965" name="l00965"></a><span class="lineno">  965</span> </div>
<div class="line"><a id="l00966" name="l00966"></a><span class="lineno">  966</span>  <span class="comment">// There could be other errors, but as in other timer handlers, we&#39;ve no idea what that means, so pretend all is OK.</span></div>
<div class="line"><a id="l00967" name="l00967"></a><span class="lineno">  967</span> </div>
<div class="line"><a id="l00968" name="l00968"></a><span class="lineno">  968</span>  <span class="comment">// Pre-condition: if we set up the timer, then the queue had to have been non-empty at the time.</span></div>
<div class="line"><a id="l00969" name="l00969"></a><span class="lineno">  969</span>  assert(!sock-&gt;m_snd_pacing_data.m_packet_q.empty());</div>
<div class="line"><a id="l00970" name="l00970"></a><span class="lineno">  970</span> </div>
<div class="line"><a id="l00971" name="l00971"></a><span class="lineno">  971</span>  <a class="code hl_define" href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a>(<span class="stringliteral">&quot;Pacing: On [&quot;</span> &lt;&lt; sock &lt;&lt; <span class="stringliteral">&quot;]: slice end timer fired; creating new slice and processing queue.&quot;</span>);</div>
<div class="line"><a id="l00972" name="l00972"></a><span class="lineno">  972</span> </div>
<div class="line"><a id="l00973" name="l00973"></a><span class="lineno">  973</span>  <span class="comment">/* Timer fired, so right now we are somewhere after the end of the current time slice.  Therefore,</span></div>
<div class="line"><a id="l00974" name="l00974"></a><span class="lineno">  974</span><span class="comment">   * begin the next time slice. */</span></div>
<div class="line"><a id="l00975" name="l00975"></a><span class="lineno">  975</span>  <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#afb88dafa8ec5c1b412de86dfdccd9c8f">sock_pacing_new_time_slice</a>(sock, Fine_clock::now());</div>
<div class="line"><a id="l00976" name="l00976"></a><span class="lineno">  976</span> </div>
<div class="line"><a id="l00977" name="l00977"></a><span class="lineno">  977</span>  <span class="comment">/* pacing.{m_slice_start, m_slice_period, m_bytes_allowed_this_slice} have all been recomputed.</span></div>
<div class="line"><a id="l00978" name="l00978"></a><span class="lineno">  978</span><span class="comment">   * By definition &quot;now&quot; is in [m_slice_start, m_slice_start + m_slice_period), since m_slice_start</span></div>
<div class="line"><a id="l00979" name="l00979"></a><span class="lineno">  979</span><span class="comment">   * IS now (plus some epsilon). */</span></div>
<div class="line"><a id="l00980" name="l00980"></a><span class="lineno">  980</span> </div>
<div class="line"><a id="l00981" name="l00981"></a><span class="lineno">  981</span>  <span class="comment">/* We are in the current time slice.  We just created the new time slice above, so</span></div>
<div class="line"><a id="l00982" name="l00982"></a><span class="lineno">  982</span><span class="comment">   * m_bytes_allowed_this_slice &gt;= max_block_size(), and certainly the following statement will</span></div>
<div class="line"><a id="l00983" name="l00983"></a><span class="lineno">  983</span><span class="comment">   * immediately send at least one packet. */</span></div>
<div class="line"><a id="l00984" name="l00984"></a><span class="lineno">  984</span> </div>
<div class="line"><a id="l00985" name="l00985"></a><span class="lineno">  985</span>  <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a73cfaa8e5ed7fe845f759a30423a9cbf">sock_pacing_process_q</a>(sock, <span class="keyword">true</span>); <span class="comment">// Process as many packets as the new budget allows.</span></div>
<div class="line"><a id="l00986" name="l00986"></a><span class="lineno">  986</span>} <span class="comment">// Node::sock_pacing_time_slice_end()</span></div>
<div class="line"><a id="l00987" name="l00987"></a><span class="lineno">  987</span> </div>
<div class="line"><a id="l00988" name="l00988"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#ad03aa88df3d1d3981454ee3356e868c1">  988</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#ad03aa88df3d1d3981454ee3356e868c1">Node::async_sock_low_lvl_rst_send</a>(<a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Peer_socket::Ptr</a> sock)</div>
<div class="line"><a id="l00989" name="l00989"></a><span class="lineno">  989</span>{</div>
<div class="line"><a id="l00990" name="l00990"></a><span class="lineno">  990</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l00991" name="l00991"></a><span class="lineno">  991</span> </div>
<div class="line"><a id="l00992" name="l00992"></a><span class="lineno">  992</span>  <span class="comment">// Fill out common fields and asynchronously send packet.</span></div>
<div class="line"><a id="l00993" name="l00993"></a><span class="lineno">  993</span>  <span class="keyword">auto</span> rst = Low_lvl_packet::create_uninit_packet_base&lt;Rst_packet&gt;(<a class="code hl_function" href="classflow_1_1log_1_1Log__context.html#af4db395042b065b00398cd59845dcb4d">get_logger</a>());</div>
<div class="line"><a id="l00994" name="l00994"></a><span class="lineno">  994</span>  <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#af1c4623729d0a061200a386f7a651926">async_sock_low_lvl_packet_send_paced</a>(sock, std::move(rst));</div>
<div class="line"><a id="l00995" name="l00995"></a><span class="lineno">  995</span>} <span class="comment">// Node::async_sock_low_lvl_rst_send()</span></div>
<div class="line"><a id="l00996" name="l00996"></a><span class="lineno">  996</span> </div>
<div class="line"><a id="l00997" name="l00997"></a><span class="lineno"><a class="line" href="classflow_1_1net__flow_1_1Node.html#a287eae1113865dba06c103cf023404ae">  997</a></span><span class="keywordtype">void</span> <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a287eae1113865dba06c103cf023404ae">Node::sync_sock_low_lvl_rst_send</a>(<a class="code hl_typedef" href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">Peer_socket::Ptr</a> sock)</div>
<div class="line"><a id="l00998" name="l00998"></a><span class="lineno">  998</span>{</div>
<div class="line"><a id="l00999" name="l00999"></a><span class="lineno">  999</span>  <span class="keyword">using </span>boost::asio::buffer;</div>
<div class="line"><a id="l01000" name="l01000"></a><span class="lineno"> 1000</span> </div>
<div class="line"><a id="l01001" name="l01001"></a><span class="lineno"> 1001</span>  <span class="comment">// We are in thread W.</span></div>
<div class="line"><a id="l01002" name="l01002"></a><span class="lineno"> 1002</span> </div>
<div class="line"><a id="l01003" name="l01003"></a><span class="lineno"> 1003</span>  <span class="comment">// Fill out fields.</span></div>
<div class="line"><a id="l01004" name="l01004"></a><span class="lineno"> 1004</span>  <span class="keyword">auto</span> rst = Low_lvl_packet::create_uninit_packet_base&lt;Rst_packet&gt;(<a class="code hl_function" href="classflow_1_1log_1_1Log__context.html#af4db395042b065b00398cd59845dcb4d">get_logger</a>());</div>
<div class="line"><a id="l01005" name="l01005"></a><span class="lineno"> 1005</span>  rst-&gt;m_packed.m_src_port = sock-&gt;m_local_port;</div>
<div class="line"><a id="l01006" name="l01006"></a><span class="lineno"> 1006</span>  rst-&gt;m_packed.m_dst_port = sock-&gt;remote_endpoint().m_flow_port;</div>
<div class="line"><a id="l01007" name="l01007"></a><span class="lineno"> 1007</span>  rst-&gt;m_opt_rexmit_on = <span class="keyword">false</span>; <span class="comment">// Unused in RST packets, so set it to something.</span></div>
<div class="line"><a id="l01008" name="l01008"></a><span class="lineno"> 1008</span> </div>
<div class="line"><a id="l01009" name="l01009"></a><span class="lineno"> 1009</span>  <span class="comment">// Serialize to a buffer sequence (basically sequence of pointers/lengths referring to existing memory areas).</span></div>
<div class="line"><a id="l01010" name="l01010"></a><span class="lineno"> 1010</span>  <a class="code hl_typedef" href="structflow_1_1net__flow_1_1Low__lvl__packet.html#a6e16e9c355c6d5e2acff5f9bdc24f397">Low_lvl_packet::Const_buffer_sequence</a> raw_bufs;</div>
<div class="line"><a id="l01011" name="l01011"></a><span class="lineno"> 1011</span>  <span class="keyword">const</span> <span class="keywordtype">size_t</span> size = rst-&gt;serialize_to_raw_data_and_log(&amp;raw_bufs); <span class="comment">// Logs TRACE/DATA.</span></div>
<div class="line"><a id="l01012" name="l01012"></a><span class="lineno"> 1012</span> </div>
<div class="line"><a id="l01013" name="l01013"></a><span class="lineno"> 1013</span>  <span class="comment">// This special-case sending path should report stats similarly to the main path elsewhere.</span></div>
<div class="line"><a id="l01014" name="l01014"></a><span class="lineno"> 1014</span>  <span class="keyword">const</span> <span class="keyword">auto</span>&amp; rst_type_id = <span class="keyword">typeid</span>(<a class="code hl_struct" href="structflow_1_1net__flow_1_1Rst__packet.html">Rst_packet</a>); <span class="comment">// ...a/k/a typeid(*rst).</span></div>
<div class="line"><a id="l01015" name="l01015"></a><span class="lineno"> 1015</span>  sock-&gt;m_snd_stats.low_lvl_packet_xfer_requested(rst_type_id);</div>
<div class="line"><a id="l01016" name="l01016"></a><span class="lineno"> 1016</span>  sock-&gt;m_snd_stats.low_lvl_packet_xfer_called(rst_type_id, <span class="keyword">false</span>, size);</div>
<div class="line"><a id="l01017" name="l01017"></a><span class="lineno"> 1017</span> </div>
<div class="line"><a id="l01018" name="l01018"></a><span class="lineno"> 1018</span>  <span class="comment">// Same check as when using async_send_to().  @todo Code reuse?</span></div>
<div class="line"><a id="l01019" name="l01019"></a><span class="lineno"> 1019</span>  <span class="keyword">const</span> <span class="keywordtype">size_t</span> limit = <a class="code hl_function" href="classflow_1_1net__flow_1_1Node.html#a43c42121d48a55a43a48e3a84bd10595">opt</a>(<a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#ade08f0457943f803c23b55649139c0d3">m_opts</a>.<a class="code hl_variable" href="structflow_1_1net__flow_1_1Node__options.html#a88c732d10914f406cc43a2c20cc800c6">m_dyn_low_lvl_max_packet_size</a>);</div>
<div class="line"><a id="l01020" name="l01020"></a><span class="lineno"> 1020</span>  <span class="keywordflow">if</span> (size &gt; limit)</div>
<div class="line"><a id="l01021" name="l01021"></a><span class="lineno"> 1021</span>  {</div>
<div class="line"><a id="l01022" name="l01022"></a><span class="lineno"> 1022</span>    <span class="comment">// Bad and rare enough for a warning.</span></div>
<div class="line"><a id="l01023" name="l01023"></a><span class="lineno"> 1023</span>    <a class="code hl_define" href="log_8hpp.html#a626c7dc4d3b4dc0b32a8aac8624d66bc">FLOW_LOG_WARNING</a>(<span class="stringliteral">&quot;Tried to send RST but before doing so detected &quot;</span></div>
<div class="line"><a id="l01024" name="l01024"></a><span class="lineno"> 1024</span>                     <span class="stringliteral">&quot;serialized size [&quot;</span> &lt;&lt; size &lt;&lt; <span class="stringliteral">&quot;] exceeds limit [&quot;</span> &lt;&lt; limit &lt;&lt; <span class="stringliteral">&quot;]; &quot;</span></div>
<div class="line"><a id="l01025" name="l01025"></a><span class="lineno"> 1025</span>                     <span class="stringliteral">&quot;check max-block-size and low-lvl-max-packet-size options!  Serialized packet: &quot;</span></div>
<div class="line"><a id="l01026" name="l01026"></a><span class="lineno"> 1026</span>                     <span class="stringliteral">&quot;[\n&quot;</span> &lt;&lt; rst-&gt;m_concise_ostream_manip &lt;&lt; <span class="stringliteral">&quot;].&quot;</span>);</div>
<div class="line"><a id="l01027" name="l01027"></a><span class="lineno"> 1027</span> </div>
<div class="line"><a id="l01028" name="l01028"></a><span class="lineno"> 1028</span>    sock-&gt;m_snd_stats.low_lvl_packet_xfer_completed(rst_type_id, size, 0); <span class="comment">// No send occurred.</span></div>
<div class="line"><a id="l01029" name="l01029"></a><span class="lineno"> 1029</span>  }</div>
<div class="line"><a id="l01030" name="l01030"></a><span class="lineno"> 1030</span>  <span class="keywordflow">else</span></div>
<div class="line"><a id="l01031" name="l01031"></a><span class="lineno"> 1031</span>  {</div>
<div class="line"><a id="l01032" name="l01032"></a><span class="lineno"> 1032</span>    <span class="comment">// Synchronously send to remote UDP.  If non-blocking mode and not sendable, this will return error.</span></div>
<div class="line"><a id="l01033" name="l01033"></a><span class="lineno"> 1033</span>    <a class="code hl_typedef" href="namespaceflow.html#a29eaaa9d0fac4ce87d8b969222dbed09">Error_code</a> sys_err_code;</div>
<div class="line"><a id="l01034" name="l01034"></a><span class="lineno"> 1034</span>    <span class="keyword">const</span> <span class="keywordtype">size_t</span> size_sent = <a class="code hl_variable" href="classflow_1_1net__flow_1_1Node.html#a9b9e0224c17190e99debdfd81640bf89">m_low_lvl_sock</a>.send_to(raw_bufs,</div>
<div class="line"><a id="l01035" name="l01035"></a><span class="lineno"> 1035</span>                                                    sock-&gt;remote_endpoint().m_udp_endpoint, 0, sys_err_code);</div>
<div class="line"><a id="l01036" name="l01036"></a><span class="lineno"> 1036</span>    <span class="keywordflow">if</span> (sys_err_code)</div>
<div class="line"><a id="l01037" name="l01037"></a><span class="lineno"> 1037</span>    {</div>
<div class="line"><a id="l01038" name="l01038"></a><span class="lineno"> 1038</span>      <a class="code hl_define" href="error_2error_8hpp.html#a019ee6e90802e444572dd78d312d6e23">FLOW_ERROR_SYS_ERROR_LOG_WARNING</a>();</div>
<div class="line"><a id="l01039" name="l01039"></a><span class="lineno"> 1039</span>      sock-&gt;m_snd_stats.low_lvl_packet_xfer_completed(rst_type_id);</div>
<div class="line"><a id="l01040" name="l01040"></a><span class="lineno"> 1040</span>    }</div>
<div class="line"><a id="l01041" name="l01041"></a><span class="lineno"> 1041</span>    <span class="keywordflow">else</span></div>
<div class="line"><a id="l01042" name="l01042"></a><span class="lineno"> 1042</span>    {</div>
<div class="line"><a id="l01043" name="l01043"></a><span class="lineno"> 1043</span>      sock-&gt;m_snd_stats.low_lvl_packet_xfer_completed(rst_type_id, size, size_sent);</div>
<div class="line"><a id="l01044" name="l01044"></a><span class="lineno"> 1044</span>    }</div>
<div class="line"><a id="l01045" name="l01045"></a><span class="lineno"> 1045</span>  }</div>
<div class="line"><a id="l01046" name="l01046"></a><span class="lineno"> 1046</span>} <span class="comment">// Node::sync_sock_low_lvl_rst_send()</span></div>
<div class="line"><a id="l01047" name="l01047"></a><span class="lineno"> 1047</span> </div>
<div class="line"><a id="l01048" name="l01048"></a><span class="lineno"><a class="line" href="structflow_1_1net__flow_1_1Send__pacing__data.html#ab7f2dc947fc3c57a6a0e93efb33c8a08"> 1048</a></span><a class="code hl_function" href="structflow_1_1net__flow_1_1Send__pacing__data.html#ab7f2dc947fc3c57a6a0e93efb33c8a08">Send_pacing_data::Send_pacing_data</a>(<a class="code hl_typedef" href="namespaceflow_1_1util.html#ac3e89a8a271b0ddc76ac2a0ce488dea4">util::Task_engine</a>* task_engine) :</div>
<div class="line"><a id="l01049" name="l01049"></a><span class="lineno"> 1049</span>  m_slice_period(0), <span class="comment">// Comment says undefined is zero() so don&#39;t leave it uninitialized.  @todo Is it really needed?</span></div>
<div class="line"><a id="l01050" name="l01050"></a><span class="lineno"> 1050</span>  m_bytes_allowed_this_slice(0),</div>
<div class="line"><a id="l01051" name="l01051"></a><span class="lineno"> 1051</span>  m_slice_timer(*task_engine)</div>
<div class="line"><a id="l01052" name="l01052"></a><span class="lineno"> 1052</span>{</div>
<div class="line"><a id="l01053" name="l01053"></a><span class="lineno"> 1053</span>  <span class="comment">// Nothing.</span></div>
<div class="line"><a id="l01054" name="l01054"></a><span class="lineno"> 1054</span>}</div>
<div class="line"><a id="l01055" name="l01055"></a><span class="lineno"> 1055</span> </div>
<div class="line"><a id="l01056" name="l01056"></a><span class="lineno"> 1056</span>} <span class="comment">// namespace flow::net_flow</span></div>
<div class="ttc" id="aclassflow_1_1log_1_1Log__context_html_a8a5fef42dc2cedf3c0d1a6e83beb2948"><div class="ttname"><a href="classflow_1_1log_1_1Log__context.html#a8a5fef42dc2cedf3c0d1a6e83beb2948">flow::log::Log_context::get_log_component</a></div><div class="ttdeci">const Component &amp; get_log_component() const</div><div class="ttdoc">Returns reference to the stored Component object, particularly as many FLOW_LOG_*() macros expect.</div><div class="ttdef"><b>Definition:</b> <a href="log_8cpp_source.html#l00222">log.cpp:222</a></div></div>
<div class="ttc" id="aclassflow_1_1log_1_1Log__context_html_af4db395042b065b00398cd59845dcb4d"><div class="ttname"><a href="classflow_1_1log_1_1Log__context.html#af4db395042b065b00398cd59845dcb4d">flow::log::Log_context::get_logger</a></div><div class="ttdeci">Logger * get_logger() const</div><div class="ttdoc">Returns the stored Logger pointer, particularly as many FLOW_LOG_*() macros expect.</div><div class="ttdef"><b>Definition:</b> <a href="log_8cpp_source.html#l00217">log.cpp:217</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a0055a25892990585561e69ceb02214d5"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a0055a25892990585561e69ceb02214d5">flow::net_flow::Node::low_lvl_packet_sent</a></div><div class="ttdeci">void low_lvl_packet_sent(Peer_socket::Ptr sock, Low_lvl_packet::Const_ptr packet, size_t bytes_expected_transferred, const Error_code &amp;sys_err_code, size_t bytes_transferred)</div><div class="ttdoc">Completion handler for async_low_lvl_packet_send_impl(); called when the packet is either successfull...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00491">low_lvl_io.cpp:491</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a0556af278dc07a43f109fbd74ac64d5e"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a0556af278dc07a43f109fbd74ac64d5e">flow::net_flow::Node::sock_pacing_new_packet_ready</a></div><div class="ttdeci">void sock_pacing_new_packet_ready(Peer_socket::Ptr sock, Low_lvl_packet::Ptr packet)</div><div class="ttdoc">async_sock_low_lvl_packet_send_paced() pacing helper: Handles a DATA or ACK packet that was just pass...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00642">low_lvl_io.cpp:642</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a08a5752bf3848f31abbe753be47c1fcf"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a08a5752bf3848f31abbe753be47c1fcf">flow::net_flow::Node::async_no_sock_low_lvl_packet_send</a></div><div class="ttdeci">void async_no_sock_low_lvl_packet_send(const util::Udp_endpoint &amp;low_lvl_remote_endpoint, Low_lvl_packet::Const_ptr packet)</div><div class="ttdoc">async_low_lvl_packet_send_impl() wrapper to call when packet is to be sent to the remote side of the ...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00307">low_lvl_io.cpp:307</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a0db2bb587e0407077b11368df075f0ff"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a0db2bb587e0407077b11368df075f0ff">flow::net_flow::Node::async_sock_low_lvl_packet_send</a></div><div class="ttdeci">void async_sock_low_lvl_packet_send(Peer_socket::Ptr sock, Low_lvl_packet::Const_ptr &amp;&amp;packet, bool delayed_by_pacing)</div><div class="ttdoc">async_low_lvl_packet_send_impl() wrapper to call when packet is to be sent to the remote side of the ...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00301">low_lvl_io.cpp:301</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a0fba26db0055a969f3869014bc5c2bfc"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a0fba26db0055a969f3869014bc5c2bfc">flow::net_flow::Node::perform_accumulated_on_recv_tasks</a></div><div class="ttdeci">void perform_accumulated_on_recv_tasks()</div><div class="ttdoc">Performs all tasks to be performed at the end of low_lvl_recv_and_handle() or async part of async_wai...</div><div class="ttdef"><b>Definition:</b> <a href="node_8cpp_source.html#l00375">node.cpp:375</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a1699dfdaf3e5ebe1b1d715a9c306d116"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a1699dfdaf3e5ebe1b1d715a9c306d116">flow::net_flow::Node::handle_incoming</a></div><div class="ttdeci">void handle_incoming(util::Blob *packet_data, const util::Udp_endpoint &amp;low_lvl_remote_endpoint)</div><div class="ttdoc">Handles a just-received, not-yet-deserialized low-level packet.</div><div class="ttdef"><b>Definition:</b> <a href="node_8cpp_source.html#l00426">node.cpp:426</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a287eae1113865dba06c103cf023404ae"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a287eae1113865dba06c103cf023404ae">flow::net_flow::Node::sync_sock_low_lvl_rst_send</a></div><div class="ttdeci">void sync_sock_low_lvl_rst_send(Peer_socket::Ptr sock)</div><div class="ttdoc">Sends an RST to the other side of the given socket, synchronously.</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00997">low_lvl_io.cpp:997</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a2983d0d8a4d4eb5c3402f63f68ef44d8"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a2983d0d8a4d4eb5c3402f63f68ef44d8">flow::net_flow::Node::mark_data_packet_sent</a></div><div class="ttdeci">void mark_data_packet_sent(Peer_socket::Ptr sock, const Sequence_number &amp;seq_num)</div><div class="ttdoc">Performs important book-keeping based on the event &quot;DATA packet was sent to destination....</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00416">low_lvl_io.cpp:416</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a3fd57a0d025a78cbeccfd8a15a983575"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a3fd57a0d025a78cbeccfd8a15a983575">flow::net_flow::Node::sock_pacing_time_slice_end</a></div><div class="ttdeci">void sock_pacing_time_slice_end(Peer_socket::Ptr sock, const Error_code &amp;sys_err_code)</div><div class="ttdoc">async_sock_low_lvl_packet_send_paced() pacing helper: If sock_pacing_process_q() ran out of the last ...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00950">low_lvl_io.cpp:950</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a43472c3277cdf5e3f28d4f4f51e3bcc0"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a43472c3277cdf5e3f28d4f4f51e3bcc0">flow::net_flow::Node::m_net_env_sim</a></div><div class="ttdeci">boost::shared_ptr&lt; Net_env_simulator &gt; m_net_env_sim</div><div class="ttdoc">The object used to simulate stuff like packet loss and latency via local means directly in the code.</div><div class="ttdef"><b>Definition:</b> <a href="node_8hpp_source.html#l03671">node.hpp:3671</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a4368247d07e930bdd07ab8d2425c6de8"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a4368247d07e930bdd07ab8d2425c6de8">flow::net_flow::Node::async_wait_latency_then_handle_incoming</a></div><div class="ttdeci">void async_wait_latency_then_handle_incoming(const Fine_duration &amp;latency, util::Blob *packet_data, const util::Udp_endpoint &amp;low_lvl_remote_endpoint)</div><div class="ttdoc">Sets up handle_incoming(packet_data, low_lvl_remote_endpoint) to be called asynchronously after a spe...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00249">low_lvl_io.cpp:249</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a43c42121d48a55a43a48e3a84bd10595"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a43c42121d48a55a43a48e3a84bd10595">flow::net_flow::Node::opt</a></div><div class="ttdeci">Opt_type opt(const Opt_type &amp;opt_val_ref) const</div><div class="ttdoc">Obtain a copy of the value of a given option in a thread-safe manner.</div><div class="ttdef"><b>Definition:</b> <a href="node_8hpp_source.html#l04138">node.hpp:4138</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a73cfaa8e5ed7fe845f759a30423a9cbf"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a73cfaa8e5ed7fe845f759a30423a9cbf">flow::net_flow::Node::sock_pacing_process_q</a></div><div class="ttdeci">void sock_pacing_process_q(Peer_socket::Ptr sock, bool executing_after_delay)</div><div class="ttdoc">async_sock_low_lvl_packet_send_paced() pacing helper: Given that we are currently in the pacing time ...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00841">low_lvl_io.cpp:841</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a74a6a82a1390a76e985593969fb47184"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a74a6a82a1390a76e985593969fb47184">flow::net_flow::Node::async_no_sock_low_lvl_rst_send</a></div><div class="ttdeci">void async_no_sock_low_lvl_rst_send(Low_lvl_packet::Const_ptr causing_packet, const util::Udp_endpoint &amp;low_lvl_remote_endpoint)</div><div class="ttdoc">Sends an RST to the given UDP endpoint in response to the given incoming low-level packet that came f...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00580">low_lvl_io.cpp:580</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a90ecbad1ff3a8d2fe4b3ad1c474f7513"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a90ecbad1ff3a8d2fe4b3ad1c474f7513">flow::net_flow::Node::handle_incoming_with_simulation</a></div><div class="ttdeci">unsigned int handle_incoming_with_simulation(util::Blob *packet_data, const util::Udp_endpoint &amp;low_lvl_remote_endpoint, bool is_sim_duplicate_packet=false)</div><div class="ttdoc">Helper for low_lvl_recv_and_handle() that calls handle_incoming() on the not-yet-deserialized low-lev...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00181">low_lvl_io.cpp:181</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a9909cd2b2a77c57da9e35fcef0f8a4e5"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a9909cd2b2a77c57da9e35fcef0f8a4e5">flow::net_flow::Node::low_lvl_recv_and_handle</a></div><div class="ttdeci">void low_lvl_recv_and_handle(Error_code sys_err_code)</div><div class="ttdoc">Handles the pre-condition that m_low_lvl_sock has a UDP packet available for reading,...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00040">low_lvl_io.cpp:40</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a9b9e0224c17190e99debdfd81640bf89"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a9b9e0224c17190e99debdfd81640bf89">flow::net_flow::Node::m_low_lvl_sock</a></div><div class="ttdeci">Udp_socket m_low_lvl_sock</div><div class="ttdoc">The UDP socket used to receive low-level packets (to assemble into application layer data) and send t...</div><div class="ttdef"><b>Definition:</b> <a href="node_8hpp_source.html#l03711">node.hpp:3711</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_a9f302fe63e36f72931e1a759b0aaaed4"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#a9f302fe63e36f72931e1a759b0aaaed4">flow::net_flow::Node::async_low_lvl_packet_send_impl</a></div><div class="ttdeci">void async_low_lvl_packet_send_impl(const util::Udp_endpoint &amp;low_lvl_remote_endpoint, Low_lvl_packet::Const_ptr packet, bool delayed_by_pacing, Peer_socket::Ptr sock)</div><div class="ttdoc">Takes given low-level packet structure, serializes it, and initiates asynchronous send of these data ...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00315">low_lvl_io.cpp:315</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_acea4db94c4fb86ddaa2502c2d7e128ea"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#acea4db94c4fb86ddaa2502c2d7e128ea">flow::net_flow::Node::m_packet_data</a></div><div class="ttdeci">util::Blob m_packet_data</div><div class="ttdoc">Stores incoming raw packet data; re-used repeatedly for possible performance gains.</div><div class="ttdef"><b>Definition:</b> <a href="node_8hpp_source.html#l03732">node.hpp:3732</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_ad03aa88df3d1d3981454ee3356e868c1"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#ad03aa88df3d1d3981454ee3356e868c1">flow::net_flow::Node::async_sock_low_lvl_rst_send</a></div><div class="ttdeci">void async_sock_low_lvl_rst_send(Peer_socket::Ptr sock)</div><div class="ttdoc">Sends an RST to the other side of the given socket asynchronously when possible.</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00988">low_lvl_io.cpp:988</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_ade08f0457943f803c23b55649139c0d3"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#ade08f0457943f803c23b55649139c0d3">flow::net_flow::Node::m_opts</a></div><div class="ttdeci">Node_options m_opts</div><div class="ttdoc">This Node's global set of options.</div><div class="ttdef"><b>Definition:</b> <a href="node_8hpp_source.html#l03662">node.hpp:3662</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_ade12d2ebbd88b62432089a3a071b9514"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#ade12d2ebbd88b62432089a3a071b9514">flow::net_flow::Node::async_low_lvl_recv</a></div><div class="ttdeci">void async_low_lvl_recv()</div><div class="ttdoc">Registers so that during the current or next m_task_engine.run(), the latter will wait for a receivab...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00031">low_lvl_io.cpp:31</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_af1c4623729d0a061200a386f7a651926"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#af1c4623729d0a061200a386f7a651926">flow::net_flow::Node::async_sock_low_lvl_packet_send_paced</a></div><div class="ttdeci">void async_sock_low_lvl_packet_send_paced(const Peer_socket::Ptr &amp;sock, Low_lvl_packet::Ptr &amp;&amp;packet)</div><div class="ttdoc">Begins the process of asynchronously sending the given low-level packet to the remote Node specified ...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00599">low_lvl_io.cpp:599</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_af5597a417d03e2bed87cd772ef5cef29"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#af5597a417d03e2bed87cd772ef5cef29">flow::net_flow::Node::m_task_engine</a></div><div class="ttdeci">util::Task_engine m_task_engine</div><div class="ttdoc">The main loop engine, functioning in the single-threaded-but-asynchronous callback-based &quot;reactor&quot; st...</div><div class="ttdef"><b>Definition:</b> <a href="node_8hpp_source.html#l03697">node.hpp:3697</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Node_html_afb88dafa8ec5c1b412de86dfdccd9c8f"><div class="ttname"><a href="classflow_1_1net__flow_1_1Node.html#afb88dafa8ec5c1b412de86dfdccd9c8f">flow::net_flow::Node::sock_pacing_new_time_slice</a></div><div class="ttdeci">void sock_pacing_new_time_slice(Peer_socket::Ptr sock, const Fine_time_pt &amp;now)</div><div class="ttdoc">async_sock_low_lvl_packet_send_paced() pacing helper: Resets the socket's Send_pacing_data structure ...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l00757">low_lvl_io.cpp:757</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Peer__socket_html_a6a9bb727c8982f24bcf978c10c2c7345a6abcbd1be8478aa03e76aec9c540e14f"><div class="ttname"><a href="classflow_1_1net__flow_1_1Peer__socket.html#a6a9bb727c8982f24bcf978c10c2c7345a6abcbd1be8478aa03e76aec9c540e14f">flow::net_flow::Peer_socket::State::S_CLOSED</a></div><div class="ttdeci">@ S_CLOSED</div><div class="ttdoc">Neither future reads nor writes are possible, AND Node has disowned the Peer_socket.</div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Peer__socket_html_a797a524b73e2626fcfefe1be64f4d481"><div class="ttname"><a href="classflow_1_1net__flow_1_1Peer__socket.html#a797a524b73e2626fcfefe1be64f4d481">flow::net_flow::Peer_socket::order_num_t</a></div><div class="ttdeci">Sequence_number::seq_num_t order_num_t</div><div class="ttdoc">Short-hand for order number type. 0 is reserved. Caution: Keep in sync with Drop_timer::packet_id_t.</div><div class="ttdef"><b>Definition:</b> <a href="peer__socket_8hpp_source.html#l00902">peer_socket.hpp:902</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Peer__socket_html_ab7dc3b752b67295c82919c2f87aed60b"><div class="ttname"><a href="classflow_1_1net__flow_1_1Peer__socket.html#ab7dc3b752b67295c82919c2f87aed60b">flow::net_flow::Peer_socket::Sent_pkt_ordered_by_when_iter</a></div><div class="ttdeci">Sent_pkt_by_sent_when_map::iterator Sent_pkt_ordered_by_when_iter</div><div class="ttdoc">Short-hand for m_snd_flying_pkts_by_sent_when iterator type.</div><div class="ttdef"><b>Definition:</b> <a href="peer__socket_8hpp_source.html#l00943">peer_socket.hpp:943</a></div></div>
<div class="ttc" id="aclassflow_1_1net__flow_1_1Sequence__number_html"><div class="ttname"><a href="classflow_1_1net__flow_1_1Sequence__number.html">flow::net_flow::Sequence_number</a></div><div class="ttdoc">An internal net_flow sequence number identifying a piece of data.</div><div class="ttdef"><b>Definition:</b> <a href="seq__num_8hpp_source.html#l00125">seq_num.hpp:126</a></div></div>
<div class="ttc" id="aclassflow_1_1util_1_1Shared__ptr__alias__holder_html_aa4f1af88393042e60159a3a67f0accbc"><div class="ttname"><a href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aa4f1af88393042e60159a3a67f0accbc">flow::util::Shared_ptr_alias_holder&lt; boost::shared_ptr&lt; Peer_socket &gt; &gt;::Ptr</a></div><div class="ttdeci">boost::shared_ptr&lt; Peer_socket &gt; Ptr</div><div class="ttdoc">Short-hand for ref-counted pointer to mutable values of type Target_type::element_type (a-la T*).</div><div class="ttdef"><b>Definition:</b> <a href="shared__ptr__alias__holder_8hpp_source.html#l00125">shared_ptr_alias_holder.hpp:125</a></div></div>
<div class="ttc" id="aclassflow_1_1util_1_1Shared__ptr__alias__holder_html_aef7998db71c60eeb5d1e3d1a97c14886"><div class="ttname"><a href="classflow_1_1util_1_1Shared__ptr__alias__holder.html#aef7998db71c60eeb5d1e3d1a97c14886">flow::util::Shared_ptr_alias_holder&lt; boost::shared_ptr&lt; Low_lvl_packet &gt; &gt;::Const_ptr</a></div><div class="ttdeci">Const_target_ptr Const_ptr</div><div class="ttdoc">Short-hand for ref-counted pointer to immutable values of type Target_type::element_type (a-la T cons...</div><div class="ttdef"><b>Definition:</b> <a href="shared__ptr__alias__holder_8hpp_source.html#l00127">shared_ptr_alias_holder.hpp:127</a></div></div>
<div class="ttc" id="acong__ctl_8hpp_html"><div class="ttname"><a href="cong__ctl_8hpp.html">cong_ctl.hpp</a></div></div>
<div class="ttc" id="adrop__timer_8hpp_html"><div class="ttname"><a href="drop__timer_8hpp.html">drop_timer.hpp</a></div></div>
<div class="ttc" id="aerror_2error_8hpp_html_a019ee6e90802e444572dd78d312d6e23"><div class="ttname"><a href="error_2error_8hpp.html#a019ee6e90802e444572dd78d312d6e23">FLOW_ERROR_SYS_ERROR_LOG_WARNING</a></div><div class="ttdeci">#define FLOW_ERROR_SYS_ERROR_LOG_WARNING()</div><div class="ttdoc">Logs a warning about the (often errno-based or from a library) error code in sys_err_code.</div><div class="ttdef"><b>Definition:</b> <a href="error_2error_8hpp_source.html#l00269">error.hpp:269</a></div></div>
<div class="ttc" id="alog_8hpp_html_a07561c1c3f16b6c2c7335a09e25298c9"><div class="ttname"><a href="log_8hpp.html#a07561c1c3f16b6c2c7335a09e25298c9">FLOW_LOG_DATA</a></div><div class="ttdeci">#define FLOW_LOG_DATA(ARG_stream_fragment)</div><div class="ttdoc">Logs a DATA message into flow::log::Logger *get_logger() with flow::log::Component get_log_component(...</div><div class="ttdef"><b>Definition:</b> <a href="log_8hpp_source.html#l00242">log.hpp:242</a></div></div>
<div class="ttc" id="alog_8hpp_html_a626c7dc4d3b4dc0b32a8aac8624d66bc"><div class="ttname"><a href="log_8hpp.html#a626c7dc4d3b4dc0b32a8aac8624d66bc">FLOW_LOG_WARNING</a></div><div class="ttdeci">#define FLOW_LOG_WARNING(ARG_stream_fragment)</div><div class="ttdoc">Logs a WARNING message into flow::log::Logger *get_logger() with flow::log::Component get_log_compone...</div><div class="ttdef"><b>Definition:</b> <a href="log_8hpp_source.html#l00152">log.hpp:152</a></div></div>
<div class="ttc" id="alog_8hpp_html_a6b3b2cd7ddabf90a7c6ccd104a042cc6"><div class="ttname"><a href="log_8hpp.html#a6b3b2cd7ddabf90a7c6ccd104a042cc6">FLOW_LOG_TRACE_WITHOUT_CHECKING</a></div><div class="ttdeci">#define FLOW_LOG_TRACE_WITHOUT_CHECKING(ARG_stream_fragment)</div><div class="ttdoc">Logs a TRACE message into flow::log::Logger *get_logger() with flow::log::Component get_log_component...</div><div class="ttdef"><b>Definition:</b> <a href="log_8hpp_source.html#l00354">log.hpp:354</a></div></div>
<div class="ttc" id="alog_8hpp_html_a9aa1d91ba9eb26345d4b83588ac489cb"><div class="ttname"><a href="log_8hpp.html#a9aa1d91ba9eb26345d4b83588ac489cb">FLOW_LOG_DATA_WITHOUT_CHECKING</a></div><div class="ttdeci">#define FLOW_LOG_DATA_WITHOUT_CHECKING(ARG_stream_fragment)</div><div class="ttdoc">Logs a DATA message into flow::log::Logger *get_logger() with flow::log::Component get_log_component(...</div><div class="ttdef"><b>Definition:</b> <a href="log_8hpp_source.html#l00372">log.hpp:372</a></div></div>
<div class="ttc" id="alog_8hpp_html_affcf2e4c496ce3f42adf981669c623b8"><div class="ttname"><a href="log_8hpp.html#affcf2e4c496ce3f42adf981669c623b8">FLOW_LOG_TRACE</a></div><div class="ttdeci">#define FLOW_LOG_TRACE(ARG_stream_fragment)</div><div class="ttdoc">Logs a TRACE message into flow::log::Logger *get_logger() with flow::log::Component get_log_component...</div><div class="ttdef"><b>Definition:</b> <a href="log_8hpp_source.html#l00227">log.hpp:227</a></div></div>
<div class="ttc" id="alow__lvl__io_8hpp_html"><div class="ttname"><a href="low__lvl__io_8hpp.html">low_lvl_io.hpp</a></div></div>
<div class="ttc" id="anamespaceflow_1_1log_html_a5c602384e6fe9d7642740ac2fae4ad04a138fe9b2eb78ab00c84b21c4a6b48fed"><div class="ttname"><a href="namespaceflow_1_1log.html#a5c602384e6fe9d7642740ac2fae4ad04a138fe9b2eb78ab00c84b21c4a6b48fed">flow::log::Sev::S_DATA</a></div><div class="ttdeci">@ S_DATA</div><div class="ttdoc">Message satisfies Sev::S_TRACE description AND contains variable-length structure (like packet,...</div></div>
<div class="ttc" id="anamespaceflow_1_1log_html_a5c602384e6fe9d7642740ac2fae4ad04a3ee808b04c34bf6c110e3754de277f7b"><div class="ttname"><a href="namespaceflow_1_1log.html#a5c602384e6fe9d7642740ac2fae4ad04a3ee808b04c34bf6c110e3754de277f7b">flow::log::Sev::S_TRACE</a></div><div class="ttdeci">@ S_TRACE</div><div class="ttdoc">Message indicates any condition that may occur with great frequency (thus verbose if logged).</div></div>
<div class="ttc" id="anamespaceflow_1_1net__flow_html"><div class="ttname"><a href="namespaceflow_1_1net__flow.html">flow::net_flow</a></div><div class="ttdoc">Flow module containing the API and implementation of the Flow network protocol, a TCP-inspired stream...</div><div class="ttdef"><b>Definition:</b> <a href="asio_2node_8cpp_source.html#l00024">node.cpp:25</a></div></div>
<div class="ttc" id="anamespaceflow_1_1util_html_a74d4a66fb00055be4e1bd6ee98e04667"><div class="ttname"><a href="namespaceflow_1_1util.html#a74d4a66fb00055be4e1bd6ee98e04667">flow::util::schedule_task_from_now</a></div><div class="ttdeci">Scheduled_task_handle schedule_task_from_now(log::Logger *logger_ptr, const Fine_duration &amp;from_now, bool single_threaded, Task_engine *task_engine, Scheduled_task_handler &amp;&amp;task_body_moved)</div><div class="ttdoc">Schedule the given function to execute in a certain amount of time: A handy wrapper around Timer (asi...</div><div class="ttdef"><b>Definition:</b> <a href="sched__task_8hpp_source.html#l00034">sched_task.hpp:34</a></div></div>
<div class="ttc" id="anamespaceflow_1_1util_html_ac3e89a8a271b0ddc76ac2a0ce488dea4"><div class="ttname"><a href="namespaceflow_1_1util.html#ac3e89a8a271b0ddc76ac2a0ce488dea4">flow::util::Task_engine</a></div><div class="ttdeci">boost::asio::io_context Task_engine</div><div class="ttdoc">Short-hand for boost.asio event service, the central class of boost.asio.</div><div class="ttdef"><b>Definition:</b> <a href="util__fwd_8hpp_source.html#l00135">util_fwd.hpp:135</a></div></div>
<div class="ttc" id="anamespaceflow_1_1util_html_af09888b3a8feaff209b6625e493bc338"><div class="ttname"><a href="namespaceflow_1_1util.html#af09888b3a8feaff209b6625e493bc338">flow::util::Blob</a></div><div class="ttdeci">Blob_with_log_context&lt;&gt; Blob</div><div class="ttdoc">A concrete Blob_with_log_context that compile-time-disables Basic_blob::share() and the sharing API d...</div><div class="ttdef"><b>Definition:</b> <a href="blob__fwd_8hpp_source.html#l00060">blob_fwd.hpp:60</a></div></div>
<div class="ttc" id="anamespaceflow_1_1util_html_af1b28edb7bad743c1c77f6b982cc3196"><div class="ttname"><a href="namespaceflow_1_1util.html#af1b28edb7bad743c1c77f6b982cc3196">flow::util::Udp_endpoint</a></div><div class="ttdeci">boost::asio::ip::udp::endpoint Udp_endpoint</div><div class="ttdoc">Short-hand for the UDP endpoint (IP/port) type.</div><div class="ttdef"><b>Definition:</b> <a href="util__fwd_8hpp_source.html#l00208">util_fwd.hpp:208</a></div></div>
<div class="ttc" id="anamespaceflow_html_a29eaaa9d0fac4ce87d8b969222dbed09"><div class="ttname"><a href="namespaceflow.html#a29eaaa9d0fac4ce87d8b969222dbed09">flow::Error_code</a></div><div class="ttdeci">boost::system::error_code Error_code</div><div class="ttdoc">Short-hand for a boost.system error code (which basically encapsulates an integer/enum error code and...</div><div class="ttdef"><b>Definition:</b> <a href="common_8hpp_source.html#l00508">common.hpp:508</a></div></div>
<div class="ttc" id="anamespaceflow_html_a48799f1263cdeedec125be51a3db2b79"><div class="ttname"><a href="namespaceflow.html#a48799f1263cdeedec125be51a3db2b79">flow::Fine_duration</a></div><div class="ttdeci">Fine_clock::duration Fine_duration</div><div class="ttdoc">A high-res time duration as computed from two Fine_time_pts.</div><div class="ttdef"><b>Definition:</b> <a href="common_8hpp_source.html#l00416">common.hpp:416</a></div></div>
<div class="ttc" id="anamespaceflow_html_a9d9cc2eeb10d398cff5591d446b763b8"><div class="ttname"><a href="namespaceflow.html#a9d9cc2eeb10d398cff5591d446b763b8">flow::Fine_time_pt</a></div><div class="ttdeci">Fine_clock::time_point Fine_time_pt</div><div class="ttdoc">A high-res time point as returned by Fine_clock::now() and suitable for precise time math in general.</div><div class="ttdef"><b>Definition:</b> <a href="common_8hpp_source.html#l00413">common.hpp:413</a></div></div>
<div class="ttc" id="anode_8hpp_html"><div class="ttname"><a href="node_8hpp.html">node.hpp</a></div></div>
<div class="ttc" id="asched__task_8hpp_html"><div class="ttname"><a href="sched__task_8hpp.html">sched_task.hpp</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Ack__packet_html"><div class="ttname"><a href="structflow_1_1net__flow_1_1Ack__packet.html">flow::net_flow::Ack_packet</a></div><div class="ttdoc">Internal net_flow struct that encapsulates the Flow-protocol low-level ACK packet.</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__packet_8hpp_source.html#l01020">low_lvl_packet.hpp:1021</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Data__packet_html"><div class="ttname"><a href="structflow_1_1net__flow_1_1Data__packet.html">flow::net_flow::Data_packet</a></div><div class="ttdoc">Internal net_flow struct that encapsulates the Flow-protocol low-level DATA packet.</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__packet_8hpp_source.html#l00893">low_lvl_packet.hpp:894</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Low__lvl__packet_html_a6e16e9c355c6d5e2acff5f9bdc24f397"><div class="ttname"><a href="structflow_1_1net__flow_1_1Low__lvl__packet.html#a6e16e9c355c6d5e2acff5f9bdc24f397">flow::net_flow::Low_lvl_packet::Const_buffer_sequence</a></div><div class="ttdeci">std::vector&lt; Const_buffer &gt; Const_buffer_sequence</div><div class="ttdoc">Short-hand for sequence of immutable buffers; i.e., a sequence of 1 or more scattered areas in memory...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__packet_8hpp_source.html#l00189">low_lvl_packet.hpp:189</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Node__options_html_a88c732d10914f406cc43a2c20cc800c6"><div class="ttname"><a href="structflow_1_1net__flow_1_1Node__options.html#a88c732d10914f406cc43a2c20cc800c6">flow::net_flow::Node_options::m_dyn_low_lvl_max_packet_size</a></div><div class="ttdeci">size_t m_dyn_low_lvl_max_packet_size</div><div class="ttdoc">Any incoming low-level (UDP) packet will be truncated to this size.</div><div class="ttdef"><b>Definition:</b> <a href="options_8hpp_source.html#l00553">options.hpp:553</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Node__options_html_a99cb19cf77ac787103ac616f4415eaf9"><div class="ttname"><a href="structflow_1_1net__flow_1_1Node__options.html#a99cb19cf77ac787103ac616f4415eaf9">flow::net_flow::Node_options::m_dyn_max_packets_per_main_loop_iteration</a></div><div class="ttdeci">unsigned int m_dyn_max_packets_per_main_loop_iteration</div><div class="ttdoc">The UDP net-stack may deliver 2 or more datagrams to the Flow Node at the same time.</div><div class="ttdef"><b>Definition:</b> <a href="options_8hpp_source.html#l00545">options.hpp:545</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Node__options_html_abef8889be34024b07eff3211a3213d5f"><div class="ttname"><a href="structflow_1_1net__flow_1_1Node__options.html#abef8889be34024b07eff3211a3213d5f">flow::net_flow::Node_options::m_st_timer_min_period</a></div><div class="ttdeci">Fine_duration m_st_timer_min_period</div><div class="ttdoc">A time period such that the boost.asio timer implementation for this platform is able to accurately a...</div><div class="ttdef"><b>Definition:</b> <a href="options_8hpp_source.html#l00534">options.hpp:534</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Peer__socket_1_1Sent__packet_html"><div class="ttname"><a href="structflow_1_1net__flow_1_1Peer__socket_1_1Sent__packet.html">flow::net_flow::Peer_socket::Sent_packet</a></div><div class="ttdoc">Metadata (and data, if retransmission is on) for a packet that has been sent one (if retransmission i...</div><div class="ttdef"><b>Definition:</b> <a href="peer__socket_8hpp_source.html#l02174">peer_socket.hpp:2178</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Peer__socket_1_1Sent__packet_html_a3acccd2fcd0f4a0c5f7f0c2d6ad31c2e"><div class="ttname"><a href="structflow_1_1net__flow_1_1Peer__socket_1_1Sent__packet.html#a3acccd2fcd0f4a0c5f7f0c2d6ad31c2e">flow::net_flow::Peer_socket::Sent_packet::m_sent_when</a></div><div class="ttdeci">std::vector&lt; Sent_when &gt; m_sent_when</div><div class="ttdoc">Time stamps, order numbers, and other info at the times when the different attempts (including origin...</div><div class="ttdef"><b>Definition:</b> <a href="peer__socket_8hpp_source.html#l02217">peer_socket.hpp:2217</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Rst__packet_html"><div class="ttname"><a href="structflow_1_1net__flow_1_1Rst__packet.html">flow::net_flow::Rst_packet</a></div><div class="ttdoc">Internal net_flow struct that encapsulates the Flow-protocol low-level RST packet.</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__packet_8hpp_source.html#l01329">low_lvl_packet.hpp:1330</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Send__pacing__data_html"><div class="ttname"><a href="structflow_1_1net__flow_1_1Send__pacing__data.html">flow::net_flow::Send_pacing_data</a></div><div class="ttdoc">The current outgoing packet pacing state, including queue of low-level packets to be sent,...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8hpp_source.html#l00176">low_lvl_io.hpp:178</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Send__pacing__data_html_a100a406213a5edf2854440397719bb91"><div class="ttname"><a href="structflow_1_1net__flow_1_1Send__pacing__data.html#a100a406213a5edf2854440397719bb91">flow::net_flow::Send_pacing_data::m_slice_timer</a></div><div class="ttdeci">util::Timer m_slice_timer</div><div class="ttdoc">When running, m_packet_q is non-empty, m_bytes_allowed_this_slice &lt; data size of m_packet_q....</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8hpp_source.html#l00232">low_lvl_io.hpp:232</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Send__pacing__data_html_a9cfd632911c3c4acd8e048dae28ded46"><div class="ttname"><a href="structflow_1_1net__flow_1_1Send__pacing__data.html#a9cfd632911c3c4acd8e048dae28ded46">flow::net_flow::Send_pacing_data::m_bytes_allowed_this_slice</a></div><div class="ttdeci">size_t m_bytes_allowed_this_slice</div><div class="ttdoc">This many bytes worth of DATA packets may still be sent, at this time, within the time slice defined ...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8hpp_source.html#l00212">low_lvl_io.hpp:212</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Send__pacing__data_html_ab7f2dc947fc3c57a6a0e93efb33c8a08"><div class="ttname"><a href="structflow_1_1net__flow_1_1Send__pacing__data.html#ab7f2dc947fc3c57a6a0e93efb33c8a08">flow::net_flow::Send_pacing_data::Send_pacing_data</a></div><div class="ttdeci">Send_pacing_data(util::Task_engine *task_engine)</div><div class="ttdoc">Initializes data to initial state (no active time slice).</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8cpp_source.html#l01048">low_lvl_io.cpp:1048</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Send__pacing__data_html_aed68c9912e8c2ada5a9fcef0f9df2331"><div class="ttname"><a href="structflow_1_1net__flow_1_1Send__pacing__data.html#aed68c9912e8c2ada5a9fcef0f9df2331">flow::net_flow::Send_pacing_data::m_packet_q</a></div><div class="ttdeci">Packet_q m_packet_q</div><div class="ttdoc">Queue of low-level packets to be sent to the remote endpoint, in order in which they are to be sent,...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8hpp_source.html#l00221">low_lvl_io.hpp:221</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Send__pacing__data_html_af0c881081c6706dba393ef814386b143"><div class="ttname"><a href="structflow_1_1net__flow_1_1Send__pacing__data.html#af0c881081c6706dba393ef814386b143">flow::net_flow::Send_pacing_data::m_slice_period</a></div><div class="ttdeci">Fine_duration m_slice_period</div><div class="ttdoc">The length of the current pacing time slice period; this depends on congestion window and SRTT on the...</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8hpp_source.html#l00199">low_lvl_io.hpp:199</a></div></div>
<div class="ttc" id="astructflow_1_1net__flow_1_1Send__pacing__data_html_af1e8f11a36db3b967d9a5f61b115750a"><div class="ttname"><a href="structflow_1_1net__flow_1_1Send__pacing__data.html#af1e8f11a36db3b967d9a5f61b115750a">flow::net_flow::Send_pacing_data::m_slice_start</a></div><div class="ttdeci">Fine_time_pt m_slice_start</div><div class="ttdoc">The time point at which the last pacing time slice began; or epoch if no packets sent so far (i....</div><div class="ttdef"><b>Definition:</b> <a href="low__lvl__io_8hpp_source.html#l00192">low_lvl_io.hpp:192</a></div></div>
</div><!-- fragment --></div><!-- contents -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Fri Mar 28 2025 22:55:35 for Flow by&#160;<a href="https://www.doxygen.org/index.html"><img class="footer" src="doxygen.svg" width="104" height="31" alt="doxygen"/></a> 1.9.4
</small></address>
</body>
</html>
